Nov 12

Nginx from the 0.7.48 release, support for Squid cache similar function. This cache is used as the URL and the relevant combination of Key, with the md5 hash code and saved on the hard drive, so it can support any URL link, also supports 404/301/302 such non-200 status code. Although the official Nginx Web caching service can only status code for the specified URL or set an expiration time, like Squid does not support the PURGE command to manually clear the cache page specified, but Nginx module by a third party, you can clear the cache of the specified URL .
Nginx’s Web caching services are mainly related to the instruction set and fastcgi_cache proxy_cache related instruction set composition, the former agent for the reverse, on the back-end content source server cache, which is mainly used to cache the FastCGI dynamic process. Both features are basically the same.
The latest version of Nginx 0.8.32, proxy_cache and fastcgi_cache been more perfect, with a third party ngx_cache_purge module (used to clear the cache of the specified URL), they can already completely replace the Squid. We have used in the production environment, the proxy_cache Nginx caching more than two months, very stable, speed is not inferior to Squid.
In function, Nginx have already have Squid Web cache acceleration, clear the cache function of the specified URL. In performance, Nginx on the use of multi-core CPU is better than a lot of Squid. In addition, the reverse proxy, load balancing, health checks, back-end server fail, Rewrite rewrite, ease of use, Nginx much stronger than the Squid. This makes a Nginx may also be a "load-balancing server" and "Web cache server" to use.

Install Part:

1.download purge patch for nginx and unzip it
wget http://labs.frickle.com/files/ngx_cache_purge-1.0.tar.gz
tar zxvf ngx_cache_purge-1.0.tar.gz

2.download nginx and unzip nginx
wget http://nginx.org/download/nginx-0.8.32.tar.gz
tar zxvf nginx-0.8.32.tar.gz

3.compile nginx
cd nginx-0.8.32/
./configure –user=www –group=www –add-module=../ngx_cache_purge-1.0 –prefix=/opt/nginx –with-http_stub_status_module –with-http_ssl_module
make && make install
cd /opt/nginx/conf

4.modify config file nginx.conf

user  www www;
worker_processes 8;
error_log  /opt/nginx/logs/nginx_error.log  crit;
pid        /opt/nginx/nginx.pid;
worker_rlimit_nofile 65535;

events
{
  use epoll;
  worker_connections 65535;
}

http
{
  include       mime.types;
  default_type  application/octet-stream;

  #charset  utf-8;

  server_names_hash_bucket_size 128;
  client_header_buffer_size 32k;
  large_client_header_buffers 4 32k;
  client_max_body_size 300m;
  sendfile on;
  tcp_nopush     on;
  keepalive_timeout 60;
  tcp_nodelay on;

  client_body_buffer_size  512k;
  proxy_connect_timeout    5;
  proxy_read_timeout       60;
  proxy_send_timeout       5;
  proxy_buffer_size        16k;
  proxy_buffers            4 64k;
  proxy_busy_buffers_size 128k;
  proxy_temp_file_write_size 128k;

#gzip on;
#gzip_min_length  1k;
#gzip_buffers     4 16k;
#gzip_http_version 1.1;
#gzip_comp_level 2;
#gzip_types       text/plain application/x-javascript text/css application/xml;
#gzip_vary on;

proxy_temp_path   /data/proxy_temp_dir;
proxy_cache_path  /data/proxy_cache_dir  levels=1:2   keys_zone=cache_one:500m inactive=1d max_size=20g;

upstream backend_server {
  server   192.168.1.38:80 weight=1 max_fails=2 fail_timeout=30s;
}

server
{
  listen       80;
  server_name  www.test.com *.test.com;
  index index.html index.htm index.php;

  location /
  {
       proxy_next_upstream http_502 http_504 error timeout invalid_header;
       proxy_cache cache_one;
       proxy_cache_valid  200 304 12h;

       proxy_cache_key $host$uri$is_args$args;
       proxy_set_header Host  $host;
       proxy_set_header X-Forwarded-For  $remote_addr;
       proxy_pass http://backend_server;
       expires      1d;
  }

  location ~ /purge(/.*)
  {
   allow            127.0.0.1;
   allow            192.168.1.253;
   deny             all;
   proxy_cache_purge    cache_one   $host$1$is_args$args;
    }

    location ~ .*\.(php|jsp|cgi)?$
    {
         proxy_set_header Host  $host;
         proxy_set_header X-Forwarded-For  $remote_addr;
         proxy_pass http://www.test.com;
    }

    access_log  off;
  }
}

5. Nginx start script nginx-start.sh
#!/bin/sh
ulimit -SHn 65535
/opt/nginx/sbin/nginx

chmod +x nginx-start.sh
./nginx-start.sh

6.Clean special url

you can visit page to delete

upload/thumb/20101101/201011011126134751_120_90.jpg  file.
http://www.test.com/purge/upload/thumb/20101101/201011011126134751_120_90.jpg

Return Result

Successful purge

Key : www.test.com/upload/thumb/20101101/201011011126134751_120_90.jpg
Path: /data/proxy_cache_dir/0/98/d0a52447df34c0d5abe1cf34b4bf0980


nginx/0.8.32


Tagged with:
Nov 30

TCP_HIT
A valid copy of the requested object was in the cache.

TCP_MISS
The requested object was not in the cache.

TCP_REFRESH_HIT
The requested object was cached but STALE. The IMS query for the object resulted in “304 not modified”.

TCP_REF_FAIL_HIT
The requested object was cached but STALE. The IMS query failed and the stale object was delivered.

TCP_REFRESH_MISS
The requested object was cached but STALE. The IMS query returned the new content.

TCP_CLIENT_REFRESH_MISS
The client issued a “no-cache” pragma, or some analogous cache control command along with the request. Thus, the cache has to refetch the object.

TCP_IMS_HIT
The client issued an IMS request for an object which was in the cache and fresh.

TCP_SWAPFAIL_MISS
The object was believed to be in the cache, but could not be accessed.

TCP_NEGATIVE_HIT
Request for a negatively cached object, e.g. “404 not found”, for which the cache believes to know that it is inaccessible. Also refer to the explainations for negative_ttl in your squid.conf file.

TCP_MEM_HIT
A valid copy of the requested object was in the cache and it was in memory, thus avoiding disk accesses.

TCP_DENIED
Access was denied for this request.

TCP_OFFLINE_HIT
The requested object was retrieved from the cache during offline mode. The offline mode never validates any object, see offline_mode in squid.conf file.

UDP_HIT
A valid copy of the requested object was in the cache.

UDP_MISS
The requested object is not in this cache.

UDP_DENIED
Access was denied for this request.

UDP_INVALID
An invalid request was received.

UDP_MISS_NOFETCH
During “-Y” startup, or during frequent failures, a cache in hit only mode will return either UDP_HIT or this code. Neighbours will thus only fetch hits.

NONE
Seen with errors and cachemgr requests. The following codes are no longer available in Squid-2:

ERR_*
Errors are now contained in the status code.

TCP_CLIENT_REFRESH
See: TCP_CLIENT_REFRESH_MISS.

TCP_SWAPFAIL
See: TCP_SWAPFAIL_MISS.

TCP_IMS_MISS
Deleted, TCP_IMS_HIT used instead.

UDP_HIT_OBJ
Hit objects are no longer available.

UDP_RELOADING
See: UDP_MISS_NOFETCH.

Tagged with:
Jul 06

How do I configure Squid to forward all requests to another cache?

FORWARDING ALL REQUESTS TO ANOTHER CACHE
If your Web proxy machine is behind a firewall, it may well be unable to connect to destination webservers. In this instance, it can be useful to setup an internal proxy and forward all requests to a machine outside the firewall or local network.
First, you need to give Squid a parent cache. Second, you need to tell Squid it can not connect directly to origin servers. This is done with three entries in your squid.conf file:

cache_peer parentcache.yourcompany.com parent 3128 0 no-query default
never_direct allow all

This forwards all requests to the parent cache. If the parent (or parents) become unreachable, any Web requests will result in a proxy error.

The alternative to this is to configure multiple parent caches – or, if your network allows it, you can allow Squid to use direct connections only when all the parents go down you:

cache_peer parent.foo.com parent 3128 0 no-query
prefer_direct off

The default behaviour of Squid in the absence of positive ICP, HTCP, etc replies is to connect to the origin server instead of using parents. The prefer_direct offdirective tells Squid to try parents first. With this configuration, if the parent cache fails or becomes unreachable, then every request will result in an error message.

MULTIPLE PARENT CACHES

Squid can be configured to use multiple parent caches, either for resilience or load-balancing. For example:

local_domain mydomain.net
cache_host parent1.foo.com parent 3128 3130 no-query round-robin
cache_host parent2.foo.com parent 3128 3130 no-query round-robin
cache_host parent3.foo.com parent 3128 3130 no-query round-robin

This configures three parent caches, which will be used in round-robin fashion. Each parent cache will receive an equal number of requests. This does not take into account load or weighting for each host.

Tagged with:
preload preload preload