wessels@squid-cache.orgYou can download the FAQ as PDF, compressed Postscript, plain text, linuxdoc SGML source or as a compressed tar of HTML.
A HIT means that the document was found in the cache. A MISS, that it wasn't found in the cache. A negative hit means that it was found in the cache, but it doesn't exist.
The hostname is the name that was requested to be resolved.
For the Flags column:
C Means positively cached.N Means negatively cached.P Means the request is pending being dispatched.D Means the request has been dispatched and we're waiting for an answer.L Means it is a locked entry because it represents a parent or sibling.The TTL column represents ``Time To Live'' (i.e., how long
the cache entry is valid). (May be negative if the document has
expired.)
The N column is the number of IP addresses from which
the cache has documents.
The rest of the line lists all the IP addresses that have been associated with that IP cache entry.
IPCache contains data for the Hostname to IP-Number mapping, and FQDNCache does it the other way round. For example:
IP Cache Contents:
Hostname Flags lstref TTL N [IP-Number]
gorn.cc.fh-lippe.de C 0 21581 1 193.16.112.73
lagrange.uni-paderborn.de C 6 21594 1 131.234.128.245
www.altavista.digital.com C 10 21299 4 204.123.2.75 ...
2/ftp.symantec.com DL 1583 -772855 0
Flags: C --> Cached
D --> Dispatched
N --> Negative Cached
L --> Locked
lstref: Time since last use
TTL: Time-To-Live until information expires
N: Count of addresses
FQDN Cache Contents:
IP-Number Flags TTL N Hostname
130.149.17.15 C -45570 1 andele.cs.tu-berlin.de
194.77.122.18 C -58133 1 komet.teuto.de
206.155.117.51 N -73747 0
Flags: C --> Cached
D --> Dispatched
N --> Negative Cached
L --> Locked
TTL: Time-To-Live until information expires
N: Count of names
This question was asked on the squid-users mailing list, to which there were three excellent replies.
You get a ``page fault'' when your OS tries to access something in memory which is actually swapped to disk. The term ``page fault'' while correct at the kernel and CPU level, is a bit deceptive to a user, as there's no actual error - this is a normal feature of operation.
Also, this doesn't necessarily mean your squid is swapping by that much. Most operating systems also implement paging for executables, so that only sections of the executable which are actually used are read from disk into memory. Also, whenever squid needs more memory, the fact that the memory was allocated will show up in the page faults.
However, if the number of faults is unusually high, and getting bigger, this could mean that squid is swapping. Another way to verify this is using a program called ``vmstat'' which is found on most UNIX platforms. If you run this as ``vmstat 5'' this will update a display every 5 seconds. This can tell you if the system as a whole is swapping a lot (see your local man page for vmstat for more information).
It is very bad for squid to swap, as every single request will be blocked until the requested data is swapped in. It is better to tweak the cache_mem and/or memory_pools setting in squid.conf, or switch to the NOVM versions of squid, than allow this to happen.
by Peter Wemm
There's two different operations at work, Paging and swapping. Paging is when individual pages are shuffled (either discarded or swapped to/from disk), while ``swapping'' generally means the entire process got sent to/from disk.
Needless to say, swapping a process is a pretty drastic event, and usually only reserved for when there's a memory crunch and paging out cannot free enough memory quickly enough. Also, there's some variation on how swapping is implemented in OS's. Some don't do it at all or do a hybrid of paging and swapping instead.
As you say, paging out doesn't necessarily involve disk IO, eg: text (code) pages are read-only and can simply be discarded if they are not used (and reloaded if/when needed). Data pages are also discarded if unmodified, and paged out if there's been any changes. Allocated memory (malloc) is always saved to disk since there's no executable file to recover the data from. mmap() memory is variable.. If it's backed from a file, it uses the same rules as the data segment of a file - ie: either discarded if unmodified or paged out.
There's also ``demand zeroing'' of pages as well that cause faults.. If you malloc memory and it calls brk()/sbrk() to allocate new pages, the chances are that you are allocated demand zero pages. Ie: the pages are not ``really'' attached to your process yet, but when you access them for the first time, the page fault causes the page to be connected to the process address space and zeroed - this saves unnecessary zeroing of pages that are allocated but never used.
The ``page faults with physical IO'' comes from the OS via getrusage(). It's highly OS dependent on what it means. Generally, it means that the process accessed a page that was not present in memory (for whatever reason) and there was disk access to fetch it. Many OS's load executables by demand paging as well, so the act of starting squid implicitly causes page faults with disk IO - however, many (but not all) OS's use ``read ahead'' and ``prefault'' heuristics to streamline the loading. Some OS's maintain ``intent queues'' so that pages can be selected as pageout candidates ahead of time. When (say) squid touches a freshly allocated demand zero page and one is needed, the OS can page out one of the candidates on the spot, causing a 'fault with physical IO' with demand zeroing of allocated memory which doesn't happen on many other OS's. (The other OS's generally put the process to sleep while the pageout daemon finds a page for it).
The meaning of ``swapping'' varies. On FreeBSD for example, swapping out is implemented as unlocking upages, kernel stack, PTD etc for aggressive pageout with the process. The only thing left of the process in memory is the 'struct proc'. The FreeBSD paging system is highly adaptive and can resort to paging in a way that is equivalent to the traditional swapping style operation (ie: entire process). FreeBSD also tries stealing pages from active processes in order to make space for disk cache. I suspect this is why setting 'memory_pools off' on the non-NOVM squids on FreeBSD is reported to work better - the VM/buffer system could be competing with squid to cache the same pages. It's a pity that squid cannot use mmap() to do file IO on the 4K chunks in it's memory pool (I can see that this is not a simple thing to do though, but that won't stop me wishing. :-).
by John Line
The comments so far have been about what paging/swapping figures mean in a ``traditional'' context, but it's worth bearing in mind that on some systems (Sun's Solaris 2, at least), the virtual memory and filesystem handling are unified and what a user process sees as reading or writing a file, the system simply sees as paging something in from disk or a page being updated so it needs to be paged out. (I suppose you could view it as similar to the operating system memory-mapping the files behind-the-scenes.)
The effect of this is that on Solaris 2, paging figures will also include file I/O. Or rather, the figures from vmstat certainly appear to include file I/O, and I presume (but can't quickly test) that figures such as those quoted by Squid will also include file I/O.
To confirm the above (which represents an impression from what I've read and observed, rather than 100% certain facts...), using an otherwise idle Sun Ultra 1 system system I just tried using cat (small, shouldn't need to page) to copy (a) one file to another, (b) a file to /dev/null, (c) /dev/zero to a file, and (d) /dev/zero to /dev/null (interrupting the last two with control-C after a while!), while watching with vmstat. 300-600 page-ins or page-outs per second when reading or writing a file (rather than a device), essentially zero in other cases (and when not cat-ing).
So ... beware assuming that all systems are similar and that paging figures represent *only* program code and data being shuffled to/from disk - they may also include the work in reading/writing all those files you were accessing...
You'll probably want to compare the number of page faults to the number of HTTP requests. If this ratio is close to, or exceeding 1, then Squid is paging too much.
This refers to ICP replies which Squid ignored, for one of these reasons: