The logs are a valuable source of information about Squid workloads and performance. The logs record not only access information, but also system configuration errors and resource consumption (eg, memory, disk space). There are several log file maintained by Squid. Some have to be explicitely activated during compile time, others can safely be deactivated during run-time.
There are a few basic points common to all log files. The time stamps logged into the log files are usually UTC seconds unless stated otherwise. The initial time stamp usually contains a millisecond extension.
If you run your Squid from the RunCache script, a file squid.out contains the Squid startup times, and also all fatal errors, e.g. as produced by an assert() failure. If you are not using RunCache, you will not see such a file.
The cache.log file contains the debug and error messages that Squid generates. If you start your Squid using the default RunCache script, or start it with the -s command line option, a copy of certain messages will go into your syslog facilities. It is a matter of personal preferences to use a separate file for the squid log data.
From the area of automatic log file analysis, the cache.log file does not have much to offer. You will usually look into this file for automated error reports, when programming Squid, testing new features, or searching for reasons of a perceived misbehaviour, etc.
The user agent log file is only maintained, if
From the user agent log file you are able to find out about distributation of browsers of your clients. Using this option in conjunction with a loaded production squid might not be the best of all ideas.
The store.log file covers the objects currently kept on disk or removed ones. As a kind of transaction log it is ususally used for debugging purposes. A definitive statement, whether an object resides on your disks is only possible after analysing the complete log file. The release (deletion) of an object may be logged at a later time than the swap out (save to disk).
The store.log file may be of interest to log file analysis which looks into the objects on your disks and the time they spend there, or how many times a hot object was accessed. The latter may be covered by another log file, too. With knowledge of the cache_dir configuration option, this log file allows for a URL to filename mapping without recursing your cache disks. However, the Squid developers recommend to treat store.log primarily as a debug file, and so should you, unless you know what you are doing.
The print format for a store log entry (one line) consists of eleven space-separated columns, compare with the storeLog() function in file src/store_log.c:
"%9d.%03d %-7s %08X %4d %9d %9d %9d %s %d/%d %s %s\n"
The timestamp when the line was logged in UTC with a millisecond fraction.
The action the object was sumitted to, compare with src/store_log.c:
The file number for the object storage file. Please note that the path to this file is calculated according to your cache_dir configuration.
A file number of FFFFFFFF denominates "memory only" objects. Any action code for such a file number refers to an object which existed only in memory, not on disk. For instance, if a RELEASE code was logged with file number FFFFFFFF, the object existed only in memory, and was released from memory.
The HTTP reply status code.
The value of the HTTP "Date: " reply header.
The value of the HTTP "Last-Modified: " reply header.
The value of the HTTP "Expires: " reply header.
The HTTP "Content-Type" major value, or "unknown" if it cannot be determined.
This column consists of two slash separated fields:
If the advertised (or expected) length is missing, it will be set to zero. If the advertised length is not zero, but not equal to the real length, the object will be realeased from the cache.
The request method for the object, e.g. GET.
The key to the object, usually the URL.
The timestamp format for the columns Date to Expires are all expressed in UTC seconds. The actual values are parsed from the HTTP reply headers. An unparsable header is represented by a value of -1, and a missing header is represented by a value of -2.
The column key usually contains just the URL of the object. Some objects though will never become public. Thus the key is said to include a unique integer number and the request method in addition to the URL.
This logfile exists for Squid-1.0 only. The format is
[date] URL peerstatus peerhost
Most log file analysis program are based on the entries in access.log. Currently, there are two file formats possible for the log file, depending on your configuration for the emulate_httpd_log option. By default, Squid will log in its native log file format. If the above option is enabled, Squid will log in the common log file format as defined by the CERN web daemon.
The common log file format contains other information than the native log file, and less. The native format contains more information for the admin interested in cache evaluation.
The Common Logfile Format is used by numerous HTTP servers. This format consists of the following seven fields:
remotehost rfc931 authuser [date] "method URL" status bytes
It is parsable by a variety of tools. The common format contains different information than the native log file format. The HTTP version is logged, which is not logged in native log file format.
The native format is different for different major versions of Squid. For Squid-1.0 it is:
time elapsed remotehost code/status/peerstatus bytes method URL
For Squid-1.1, the information from the hierarchy.log was moved into access.log. The format is:
time elapsed remotehost code/status bytes method URL rfc931 peerstatus/peerhost type
For Squid-2 the columns stay the same, though the content within may change a little.
The native log file format logs more and different information than the common log file format: the request duration, some timeout information, the next upstream server address, and the content type.
There exist tools, which convert one file format into the other. Please mind that even though the log formats share most information, both formats contain information which is not part of the other format, and thus this part of the information is lost when converting. Especially converting back and forth is not possible without loss.
squid2common.pl is a conversion utility, which converts any of the squid log file formats into the old CERN proxy style output. There exist tools to analyse, evaluate and graph results from that format.
It is recommended though to use Squid's native log format due to its greater amount of information made available for later analysis. The print format line for native access.log entries looks like this:
"%9d.%03d %6d %s %s/%03d %d %s %s %s %s%s/%s %s"
Therefore, an access.log entry usually consists of (at least) 10 columns separated by one ore more spaces:
A Unix timestamp as UTC seconds with a millisecond resolution. You can convert Unix timestamps into something more human readable using this short perl script:
#! /usr/bin/perl -p
s/^\d+\.\d+/localtime ! a matter of personal
preferences to use a separate file for the squid log data.
From the area of automatic log file analysis, the cache.log file does
not have much to offer. You will usually look into this file for automated
error reports, when programming Squid, testing new features, or searching
for reasons of a perceived misbehaviour, etc.
6.3 useragent.log
The user agent log file is only maintained, if
- you configured the compile time --enable-useragent-log
option, and
- you pointed the useragent_log configuration option to a
file.
From the user agent log file you are able to find out about distributation
of browsers of your clients. Using this option in conjunction with a loaded
production squid might not be the best of all ideas.
6.4 store.log
The store.log file covers the objects currently kept on disk or
removed ones. As a kind of transaction log it is ususally used for
debugging purposes. A definitive statement, whether an object resides on
your disks is only possible after analysing the complete log file.
The release (deletion) of an object may be logged at a later time than the
swap out (save to disk).
The store.log file may be of interest to log file analysis which
looks into the objects on your disks and the time they spend there, or how
many times a hot object was accessed. The latter may be covered by another
log file, too. With knowledge of the cache_dir configuration option,
this log file allows for a URL to filename mapping without recursing your
cache disks. However, the Squid developers recommend to treat
store.log primarily as a debug file, and so should you, unless you
know what you are doing.
The print format for a store log entry (one line) consists of eleven
space-separated columns, compare with the storeLog() function in file
src/store_log.c:
"%9d.%03d %-7s %08X %4d %9d %9d %9d %s %d/%d %s %s\n"
The timestamp when the line was logged in UTC with a millisecond fraction.
The action the object was sumitted to, compare with src/store_log.c:
The file number for the object storage file. Please note that the path to this file is calculated according to your cache_dir configuration.
A file number of FFFFFFFF denominates "memory only" objects. Any action code for such a file number refers to an object which existed only in memory, not on disk. For instance, if a RELEASE code was logged with file number FFFFFFFF, the object existed only in memory, and was released from memory.
The HTTP reply status code.
The value of the HTTP "Date: " reply header.
The value of the HTTP "Last-Modified: " reply header.
The value of the HTTP "Expires: " reply header.
The HTTP "Content-Type" major value, or "unknown" if it cannot be determined.
This column consists of two slash separated fields:
If the advertised (or expected) length is missing, it will be set to zero. If the advertised length is not zero, but not equal to the real length, the object will be realeased from the cache.
The request method for the object, e.g. GET.
The key to the object, usually the URL.
The timestamp format for the columns Date to Expires are all expressed in UTC seconds. The actual values are parsed from the HTTP reply headers. An unparsable header is represented by a value of -1, and a missing header is represented by a value of -2.
The column key usually contains just the URL of the object. Some objects though will never become public. Thus the key is said to include a unique integer number and the request method in addition to the URL.
This logfile exists for Squid-1.0 only. The format is
[date] URL peerstatus peerhost
Most log file analysis program are based on the entries in access.log. Currently, there are two file formats possible for the log file, depending on your configuration for the emulate_httpd_log option. By default, Squid will log in its native log file format. If the above option is enabled, Squid will log in the common log file format as defined by the CERN web daemon.
The common log file format contains other information than the native log file, and less. The native format contains more information for the admin interested in cache evaluation.
The Common Logfile Format is used by numerous HTTP servers. This format consists of the following seven fields:
remotehost rfc931 authuser [date] "method URL" status bytes
It is parsable by a variety of tools. The common format contains different information than the native log file format. The HTTP version is logged, which is not logged in native log file format.
The native format is different for different major versions of Squid. For Squid-1.0 it is:
time elapsed remotehost code/status/peerstatus bytes method URL
For Squid-1.1, the information from the hierarchy.log was moved into access.log. The format is:
time elapsed remotehost code/status bytes method URL rfc931 peerstatus/peerhost type
For Squid-2 the columns stay the same, though the content within may change a little.
The native log file format logs more and different information than the common log file format: the request duration, some timeout information, the next upstream server address, and the content type.
There exist tools, which convert one file format into the other. Please mind that even though the log formats share most information, both formats contain information which is not part of the other format, and thus this part of the information is lost when converting. Especially converting back and forth is not possible without loss.
squid2common.pl is a conversion utility, which converts any of the squid log file formats into the old CERN proxy style output. There exist tools to analyse, evaluate and graph results from that format.
It is recommended though to use Squid's native log format due to its greater amount of information made available for later analysis. The print format line for native access.log entries looks like this:
"%9d.%03d %6d %s %s/%03d %d %s %s %s %s%s/%s %s"
Therefore, an access.log entry usually consists of (at least) 10 columns separated by one ore more spaces:
A Unix timestamp as UTC seconds with a millisecond resolution. You can convert Unix timestamps into something more human readable using this short perl script:
#! /usr/bin/perl -p
s/^\d+\.\d+/localtime ! a matter of personal
preferences to use a separate file for the squid log data.
From the area of automatic log file analysis, the cache.log file does
not have much to offer. You will usually look into this file for automated
error reports, when programming Squid, testing new features, or searching
for reasons of a perceived misbehaviour, etc.
6.3 useragent.log
The user agent log file is only maintained, if
- you configured the compile time --enable-useragent-log
option, and
- you pointed the useragent_log configuration option to a
file.
From the user agent log file you are able to find out about distributation
of browsers of your clients. Using this option in conjunction with a loaded
production squid might not be the best of all ideas.
6.4 store.log
The store.log file covers the objects currently kept on disk or
removed ones. As a kind of transaction log it is ususally used for
debugging purposes. A definitive statement, whether an object resides on
your disks is only possible after analysing the complete log file.
The release (deletion) of an object may be logged at a later time than the
swap out (save to disk).
The store.log file may be of interest to log file analysis which
looks into the objects on your disks and the time they spend there, or how
many times a hot object was accessed. The latter may be covered by another
log file, too. With knowledge of the cache_dir configuration option,
this log file allows for a URL to filename mapping without recursing your
cache disks. However, the Squid developers recommend to treat
store.log primarily as a debug file, and so should you, unless you
know what you are doing.
The print format for a store log entry (one line) consists of eleven
space-separated columns, compare with the storeLog() function in file
src/store_log.c:
"%9d.%03d %-7s %08X %4d %9d %9d %9d %s %d/%d %s %s\n"
The timestamp when the line was logged in UTC with a millisecond fraction.
The action the object was sumitted to, compare with src/store_log.c:
The file number for the object storage file. Please note that the path to this file is calculated according to your cache_dir configuration.
A file number of FFFFFFFF denominates "memory only" objects. Any action code for such a file number refers to an object which existed only in memory, not on disk. For instance, if a RELEASE code was logged with file number FFFFFFFF, the object existed only in memory, and was released from memory.
The HTTP reply status code.
The value of the HTTP "Date: " reply header.
The value of the HTTP "Last-Modified: " reply header.
The value of the HTTP "Expires: " reply header.
The HTTP "Content-Type" major value, or "unknown" if it cannot be determined.
This column consists of two slash separated fields:
If the advertised (or expected) length is missing, it will be set to zero. If the advertised length is not zero, but not equal to the real length, the object will be realeased from the cache.
The request method for the object, e.g. GET.
The key to the object, usually the URL.
The timestamp format for the columns Date to Expires are all expressed in UTC seconds. The actual values are parsed from the HTTP reply headers. An unparsable header is represented by a value of -1, and a missing header is represented by a value of -2.
The column key usually contains just the URL of the object. Some objects though will never become public. Thus the key is said to include a unique integer number and the request method in addition to the URL.
This logfile exists for Squid-1.0 only. The format is
[date] URL peerstatus peerhost
Most log file analysis program are based on the entries in access.log. Currently, there are two file formats possible for the log file, depending on your configuration for the emulate_httpd_log option. By default, Squid will log in its native log file format. If the above option is enabled, Squid will log in the common log file format as defined by the CERN web daemon.
The common log file format contains other information than the native log file, and less. The native format contains more information for the admin interested in cache evaluation.
The Common Logfile Format is used by numerous HTTP servers. This format consists of the following seven fields:
remotehost rfc931 authuser [date] "method URL" status bytes
It is parsable by a variety of tools. The common format contains different information than the native log file format. The HTTP version is logged, which is not logged in native log file format.
The native format is different for different major versions of Squid. For Squid-1.0 it is:
time elapsed remotehost code/status/peerstatus bytes method URL
For Squid-1.1, the information from the hierarchy.log was moved into access.log. The format is:
time elapsed remotehost code/status bytes method URL rfc931 peerstatus/peerhost type
For Squid-2 the columns stay the same, though the content within may change a little.
The native log file format logs more and different information than the common log file format: the request duration, some timeout information, the next upstream server address, and the content type.
There exist tools, which convert one file format into the other. Please mind that even though the log formats share most information, both formats contain information which is not part of the other format, and thus this part of the information is lost when converting. Especially converting back and forth is not possible without loss.
squid2common.pl is a conversion utility, which converts any of the squid log file formats into the old CERN proxy style output. There exist tools to analyse, evaluate and graph results from that format.
It is recommended though to use Squid's native log format due to its greater amount of information made available for later analysis. The print format line for native access.log entries looks like this:
"%9d.%03d %6d %s %s/%03d %d %s %s %s %s%s/%s %s"
Therefore, an access.log entry usually consists of (at least) 10 columns separated by one ore more spaces:
A Unix timestamp as UTC seconds with a millisecond resolution. You can convert Unix timestamps into something more human readable using this short perl script:
#! /usr/bin/perl -p
s/^\d+\.\d+/localtime ! a matter of personal
preferences to use a separate file for the squid log data.
From the area of automatic log file analysis, the cache.log file does
not have much to offer. You will usually look into this file for automated
error reports, when programming Squid, testing new features, or searching
for reasons of a perceived misbehaviour, etc.
6.3 useragent.log
The user agent log file is only maintained, if
- you configured the compile time --enable-useragent-log
option, and
- you pointed the useragent_log configuration option to a
file.
From the user agent log file you are able to find out about distributation
of browsers of your clients. Using this option in conjunction with a loaded
production squid might not be the best of all ideas.
6.4 store.log
The store.log file covers the objects currently kept on disk or
removed ones. As a kind of transaction log it is ususally used for
debugging purposes. A definitive statement, whether an object resides on
your disks is only possible after analysing the complete log file.
The release (deletion) of an object may be logged at a later time than the
swap out (save to disk).
The store.log file may be of interest to log file analysis which
looks into the objects on your disks and the time they spend there, or how
many times a hot object was accessed. The latter may be covered by another
log file, too. With knowledge of the cache_dir configuration option,
this log file allows for a URL to filename mapping without recursing your
cache disks. However, the Squid developers recommend to treat
store.log primarily as a debug file, and so should you, unless you
know what you are doing.
The print format for a store log entry (one line) consists of eleven
space-separated columns, compare with the storeLog() function in file
src/store_log.c:
"%9d.%03d %-7s %08X %4d %9d %9d %9d %s %d/%d %s %s\n"
The timestamp when the line was logged in UTC with a millisecond fraction.
The action the object was sumitted to, compare with src/store_log.c:
The file number for the object storage file. Please note that the path to this file is calculated according to your cache_dir configuration.
A file number of FFFFFFFF denominates "memory only" objects. Any action code for such a file number refers to an object which existed only in memory, not on disk. For instance, if a RELEASE code was logged with file number FFFFFFFF, the object existed only in memory, and was released from memory.
The HTTP reply status code.
The value of the HTTP "Date: " reply header.
The value of the HTTP "Last-Modified: " reply header.
The value of the HTTP "Expires: " reply header.
The HTTP "Content-Type" major value, or "unknown" if it cannot be determined.
This column consists of two slash separated fields:
If the advertised (or expected) length is missing, it will be set to zero. If the advertised length is not zero, but not equal to the real length, the object will be realeased from the cache.
The request method for the object, e.g. GET.
The key to the object, usually the URL.
The timestamp format for the columns Date to Expires are all expressed in UTC seconds. The actual values are parsed from the HTTP reply headers. An unparsable header is represented by a value of -1, and a missing header is represented by a value of -2.
The column key usually contains just the URL of the object. Some objects though will never become public. Thus the key is said to include a unique integer number and the request method in addition to the URL.
This logfile exists for Squid-1.0 only. The format is
[date] URL peerstatus peerhost
Most log file analysis program are based on the entries in access.log. Currently, there are two file formats possible for the log file, depending on your configuration for the emulate_httpd_log option. By default, Squid will log in its native log file format. If the above option is enabled, Squid will log in the common log file format as defined by the CERN web daemon.
The common log file format contains other information than the native log file, and less. The native format contains more information for the admin interested in cache evaluation.
The Common Logfile Format is used by numerous HTTP servers. This format consists of the following seven fields:
remotehost rfc931 authuser [date] "method URL" status bytes
It is parsable by a variety of tools. The common format contains different information than the native log file format. The HTTP version is logged, which is not logged in native log file format.
The native format is different for different major versions of Squid. For Squid-1.0 it is:
time elapsed remotehost code/status/peerstatus bytes method URL
For Squid-1.1, the information from the hierarchy.log was moved into access.log. The format is:
time elapsed remotehost code/status bytes method URL rfc931 peerstatus/peerhost type
For Squid-2 the columns stay the same, though the content within may change a little.
The native log file format logs more and different information than the common log file format: the request duration, some timeout information, the next upstream server address, and the content type.
There exist tools, which convert one file format into the other. Please mind that even though the log formats share most information, both formats contain information which is not part of the other format, and thus this part of the information is lost when converting. Especially converting back and forth is not possible without loss.
squid2common.pl is a conversion utility, which converts any of the squid log file formats into the old CERN proxy style output. There exist tools to analyse, evaluate and graph results from that format.
It is recommended though to use Squid's native log format due to its greater amount of information made available for later analysis. The print format line for native access.log entries looks like this:
"%9d.%03d %6d %s %s/%03d %d %s %s %s %s%s/%s %s"
Therefore, an access.log entry usually consists of (at least) 10 columns separated by one ore more spaces:
A Unix timestamp as UTC seconds with a millisecond resolution. You can convert Unix timestamps into something more human readable using this short perl script:
#! /usr/bin/perl -p
s/^\d+\.\d+/localtime ! a matter of personal
preferences to use a separate file for the squid log data.
From the area of automatic log file analysis, the cache.log file does
not have much to offer. You will usually look into this file for automated
error reports, when programming Squid, testing new features, or searching
for reasons of a perceived misbehaviour, etc.
6.3 useragent.log
The user agent log file is only maintained, if
- you configured the compile time --enable-useragent-log
option, and
- you pointed the useragent_log configuration option to a
file.
From the user agent log file you are able to find out about distributation
of browsers of your clients. Using this option in conjunction with a loaded
production squid might not be the best of all ideas.
6.4 store.log
The store.log file covers the objects currently kept on disk or
removed ones. As a kind of transaction log it is ususally used for
debugging purposes. A definitive statement, whether an object resides on
your disks is only possible after analysing the complete log file.
The release (deletion) of an object may be logged at a later time than the
swap out (save to disk).
The store.log file may be of interest to log file analysis which
looks into the objects on your disks and the time they spend there, or how
many times a hot object was accessed. The latter may be covered by another
log file, too. With knowledge of the cache_dir con