Oct 05

Modify nginx config file

location ~ \.php($|/) {
root /var/www/html;
fastcgi_pass 127.0.0.1:9000;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME /var/www/html$fastcgi_script_name;
fastcgi_split_path_info ^(.+\.php)(.*)$;
fastcgi_param PATH_INFO $fastcgi_path_info;
include fastcgi_params;
}

Modify codeigniter config

Before:
//$config['uri_protocol']       = "AUTO";
After:
$config['uri_protocol'] = "PATH_INFO";

Tagged with:
Jun 01

If your web server is apache ,you can add follow line:

<Directory /website/attachments>
php_flag engine off
</Directory>

We want disable upload directory php execute privileges on nginx,It’s so simple

location /upload/ {
location ~ .*\.(php)?$
{
deny all;
}
}

limit more directory

location ~* ^/(upload|images)/.*\.(php|php5)$
{
deny all;
}

If your web server is lighthttpd you can

$HTTP["url"] =~ "^/(forumdata|templates|customavatars?)/" {
fastcgi.server = ()
}
Apache:
<Location "/forumdata">
php_admin_flag engine off
Options -ExecCGI
AddType text/plain .html .htm .shtml .php
</Location>

Hope it will help you.

Tagged with:
Nov 12

Nginx from the 0.7.48 release, support for Squid cache similar function. This cache is used as the URL and the relevant combination of Key, with the md5 hash code and saved on the hard drive, so it can support any URL link, also supports 404/301/302 such non-200 status code. Although the official Nginx Web caching service can only status code for the specified URL or set an expiration time, like Squid does not support the PURGE command to manually clear the cache page specified, but Nginx module by a third party, you can clear the cache of the specified URL .
Nginx’s Web caching services are mainly related to the instruction set and fastcgi_cache proxy_cache related instruction set composition, the former agent for the reverse, on the back-end content source server cache, which is mainly used to cache the FastCGI dynamic process. Both features are basically the same.
The latest version of Nginx 0.8.32, proxy_cache and fastcgi_cache been more perfect, with a third party ngx_cache_purge module (used to clear the cache of the specified URL), they can already completely replace the Squid. We have used in the production environment, the proxy_cache Nginx caching more than two months, very stable, speed is not inferior to Squid.
In function, Nginx have already have Squid Web cache acceleration, clear the cache function of the specified URL. In performance, Nginx on the use of multi-core CPU is better than a lot of Squid. In addition, the reverse proxy, load balancing, health checks, back-end server fail, Rewrite rewrite, ease of use, Nginx much stronger than the Squid. This makes a Nginx may also be a "load-balancing server" and "Web cache server" to use.

Install Part:

1.download purge patch for nginx and unzip it
wget http://labs.frickle.com/files/ngx_cache_purge-1.0.tar.gz
tar zxvf ngx_cache_purge-1.0.tar.gz

2.download nginx and unzip nginx
wget http://nginx.org/download/nginx-0.8.32.tar.gz
tar zxvf nginx-0.8.32.tar.gz

3.compile nginx
cd nginx-0.8.32/
./configure –user=www –group=www –add-module=../ngx_cache_purge-1.0 –prefix=/opt/nginx –with-http_stub_status_module –with-http_ssl_module
make && make install
cd /opt/nginx/conf

4.modify config file nginx.conf

user  www www;
worker_processes 8;
error_log  /opt/nginx/logs/nginx_error.log  crit;
pid        /opt/nginx/nginx.pid;
worker_rlimit_nofile 65535;

events
{
  use epoll;
  worker_connections 65535;
}

http
{
  include       mime.types;
  default_type  application/octet-stream;

  #charset  utf-8;

  server_names_hash_bucket_size 128;
  client_header_buffer_size 32k;
  large_client_header_buffers 4 32k;
  client_max_body_size 300m;
  sendfile on;
  tcp_nopush     on;
  keepalive_timeout 60;
  tcp_nodelay on;

  client_body_buffer_size  512k;
  proxy_connect_timeout    5;
  proxy_read_timeout       60;
  proxy_send_timeout       5;
  proxy_buffer_size        16k;
  proxy_buffers            4 64k;
  proxy_busy_buffers_size 128k;
  proxy_temp_file_write_size 128k;

#gzip on;
#gzip_min_length  1k;
#gzip_buffers     4 16k;
#gzip_http_version 1.1;
#gzip_comp_level 2;
#gzip_types       text/plain application/x-javascript text/css application/xml;
#gzip_vary on;

proxy_temp_path   /data/proxy_temp_dir;
proxy_cache_path  /data/proxy_cache_dir  levels=1:2   keys_zone=cache_one:500m inactive=1d max_size=20g;

upstream backend_server {
  server   192.168.1.38:80 weight=1 max_fails=2 fail_timeout=30s;
}

server
{
  listen       80;
  server_name  www.test.com *.test.com;
  index index.html index.htm index.php;

  location /
  {
       proxy_next_upstream http_502 http_504 error timeout invalid_header;
       proxy_cache cache_one;
       proxy_cache_valid  200 304 12h;

       proxy_cache_key $host$uri$is_args$args;
       proxy_set_header Host  $host;
       proxy_set_header X-Forwarded-For  $remote_addr;
       proxy_pass http://backend_server;
       expires      1d;
  }

  location ~ /purge(/.*)
  {
   allow            127.0.0.1;
   allow            192.168.1.253;
   deny             all;
   proxy_cache_purge    cache_one   $host$1$is_args$args;
    }

    location ~ .*\.(php|jsp|cgi)?$
    {
         proxy_set_header Host  $host;
         proxy_set_header X-Forwarded-For  $remote_addr;
         proxy_pass http://www.test.com;
    }

    access_log  off;
  }
}

5. Nginx start script nginx-start.sh
#!/bin/sh
ulimit -SHn 65535
/opt/nginx/sbin/nginx

chmod +x nginx-start.sh
./nginx-start.sh

6.Clean special url

you can visit page to delete

upload/thumb/20101101/201011011126134751_120_90.jpg  file.
http://www.test.com/purge/upload/thumb/20101101/201011011126134751_120_90.jpg

Return Result

Successful purge

Key : www.test.com/upload/thumb/20101101/201011011126134751_120_90.jpg
Path: /data/proxy_cache_dir/0/98/d0a52447df34c0d5abe1cf34b4bf0980


nginx/0.8.32


Tagged with:
Nov 10

When running PHP as CGI binary on nginx. You might get above error if you nonexistent PHP file. If you got a custom 404 page,this can be irritating, as it makes for an inconsistent user experience.

I find many articles, and I check php.ini file

cgi.fix_pathinfo=1

doc_root=

And check nginx.conf config file. virtual host parameters.

fastcgi_param SCRIPT_FILENAME /var/www/html/$fastcgi_script_name;

The red part default is /script/$fastcgi_script_name;

I have modify above these parameters but if I get the  error still. I find the  parameters,I modify nginx config file add the follow a line

        fastcgi_intercept_errors on; 

After I add the parameters nginx return a custom 404 page,and not return “No input file specified” error.

I modify php.ini file

cgi.fix_pathinfo=0

the parameters have a serious security problem.

Tagged with:
Jun 18

configure determines the features of system and, in particular, the methods, which nginx can use for handling connections. Finally it creates the Makefile.

configure supports the following options:

–prefix=<path> – The path relative to which all other Nginx paths will resolve. If not specified, defaults to /usr/local/nginx.

–sbin-path=<path> – The path to the nginx executable. Only used for installation. If not specified defaults to <prefix>/sbin/nginx.

–conf-path=<path> – The default location of nginx.conf if no -c parameter is provided. If not provided, defaults to <prefix>/conf/nginx.conf.

–pid-path=<path> – The path to nginx.pid, if not set via the "pid" directive in nginx.conf. If not provided, defaults to <prefix>/logs/nginx.pid.

–lock-path=<path> – The path to the nginx.lock file. If not provided, defaults to <prefix>/logs/nginx.lock.

–error-log-path=<path> – The location of the error log if not set via the "error_log" in nginx.conf. If not set, defaults to <prefix>/logs/error.log.

–http-log-path=<path> – The location of the access log if not set via the "access_log" directive in nginx.conf. If not set, defaults to <prefix>/logs/access.log.

–user=<user> – The default user that nginx will run as if not set in nginx.conf via the "user" directive. If not set, defaults to "nobody".

–group=<group> – The default group that nginx will run under if not set via the "user" directive in nginx.conf. If not set defaults to "nobody".

–builddir=DIR – Set the build directory

–with-rtsig_module – Enable rtsig module

–with-select_module –without-select_module – Whether or not to enable the select module. This module is enabled by default if a more suitable method such as kqueue, epoll, rtsig or /dev/poll is not discovered by configure.

–with-poll_module –without-poll_module – Whether or not to enable the poll module. This module is enabled by default if a more suitable method such as kqueue, epoll, rtsig or /dev/poll is not discovered by configure.

–with-http_ssl_module – Enable ngx_http_ssl_module. Enables SSL support and the ability to handle HTTPS requests. Requires OpenSSL. On Debian, this is libssl-dev.

–with-http_realip_module – Enable ngx_http_realip_module

–with-http_addition_module – Enable ngx_http_addition_module

–with-http_sub_module – Enable ngx_http_sub_module

–with-http_dav_module – Enable ngx_http_dav_module

–with-http_flv_module – Enable ngx_http_flv_module

–with-http_stub_status_module – Enable the "server status" page

–without-http_charset_module – Disable ngx_http_charset_module

–without-http_gzip_module – Disable ngx_http_gzip_module. Requires zlib if enabled.

–without-http_ssi_module – Disable ngx_http_ssi_module

–without-http_userid_module – Disable ngx_http_userid_module

–without-http_access_module – Disable ngx_http_access_module

–without-http_auth_basic_module – Disable ngx_http_auth_basic_module

–without-http_autoindex_module – Disable ngx_http_autoindex_module

–without-http_geo_module – Disable ngx_http_geo_module

–without-http_map_module – Disable ngx_http_map_module

–without-http_referer_module – Disable ngx_http_referer_module

–without-http_rewrite_module – Disable ngx_http_rewrite_module. Requires PCRE if enabled.

–without-http_proxy_module – Disable ngx_http_proxy_module

–without-http_fastcgi_module – Disable ngx_http_fastcgi_module

–without-http_memcached_module – Disable ngx_http_memcached_module

–without-http_limit_zone_module – Disable ngx_http_limit_zone_module

–without-http_empty_gif_module – Disable ngx_http_empty_gif_module

–without-http_browser_module – Disable ngx_http_browser_module

–without-http_upstream_ip_hash_module – Disable ngx_http_upstream_ip_hash_module

–with-http_perl_module – Enable ngx_http_perl_module

–with-perl_modules_path=PATH – Set path to the perl modules

–with-perl=PATH – Set path to the perl binary

–http-client-body-temp-path=PATH – Set path to the http client request body temporary files. If not set, defaults to <prefix>/client_body_temp

–http-proxy-temp-path=PATH – Set path to the http proxy temporary files. If not set, defaults to <prefix>/proxy_temp

–http-fastcgi-temp-path=PATH – Set path to the http fastcgi temporary files. If not set, defaults to <prefix>/fastcgi_temp

–without-http – Disable HTTP server

–with-mail – Enable IMAP4/POP3/SMTP proxy module

–with-mail_ssl_module – Enable ngx_mail_ssl_module

–with-cc=PATH – Set path to C compiler

–with-cpp=PATH – Set path to C preprocessor

–with-cc-opt=OPTIONS – Additional parameters which will be added to the variable CFLAGS. With the use of the system library PCRE in FreeBSD, it is necessary to indicate –with-cc-opt="-I /usr/local/include". If we are using select() and it is necessary to increase the number of file descriptors, then this also can be assigned here: –with-cc-opt="-D FD_SETSIZE=2048".

–with-ld-opt=OPTIONS – Additional parameters passed to the linker. With the use of the system library PCRE in FreeBSD, it is necessary to indicate –with-ld-opt="-L /usr/local/lib".

–with-cpu-opt=CPU – Build for specified CPU, the valid values: pentium, pentiumpro, pentium3, pentium4, athlon, opteron, amd64, sparc32, sparc64, ppc64

–without-pcre – Disable PCRE library usage. Also disables HTTP rewrite module. PCRE is also required for regular expressions in "location" directive.

–with-pcre=DIR – Set path to PCRE library sources.

–with-pcre-opt=OPTIONS – Set additional options for PCRE building.

–with-md5=DIR – Set path to md5 library sources.

–with-md5-opt=OPTIONS – Set additional options for md5 building.

–with-md5-asm – Use md5 assembler sources.

–with-sha1=DIR – Set path to sha1 library sources.

–with-sha1-opt=OPTIONS – Set additional options for sha1 building.

–with-sha1-asm – Use sha1 assembler sources.

–with-zlib=DIR – Set path to zlib library sources.

–with-zlib-opt=OPTIONS – Set additional options for zlib building.

–with-zlib-asm=CPU – Use zlib assembler sources optimized for specified CPU, valid values are: pentium, pentiumpro

–with-openssl=DIR – Set path to OpenSSL library sources

–with-openssl-opt=OPTIONS – Set additional options for OpenSSL building

–with-debug – Enable debug logging

–add-module=PATH – Add in a third-party module found in directory PATH

Options may vary slightly between versions. Always check ./configure –help for the current list.

Edit section: Examples Examples


Edit section: Example 1 Example 1

This is a single line!

./configure \
  --sbin-path=/usr/local/nginx/nginx \
  --conf-path=/usr/local/nginx/nginx.conf \
  --pid-path=/usr/local/nginx/nginx.pid \
  --with-http_ssl_module \
  --with-pcre=../pcre-4.4 \
  --with-zlib=../zlib-1.1.3

Edit section: Example 2 Example 2

Default Debian Lenny.

./configure \
  --conf-path=/etc/nginx/nginx.conf \
  --error-log-path=/var/log/nginx/error.log \
  --pid-path=/var/run/nginx.pid \
  --lock-path=/var/lock/nginx.lock \
  --http-log-path=/var/log/nginx/access.log \
  --with-http_dav_module \
  --http-client-body-temp-path=/var/lib/nginx/body \
  --with-http_ssl_module \
  --http-proxy-temp-path=/var/lib/nginx/proxy \
  --with-http_stub_status_module \
  --http-fastcgi-temp-path=/var/lib/nginx/fastcgi \
  --with-debug \
  --with-http_flv_module 

Edit section: Example 3 Example 3

I use this configuration for 50+ worth millions impressions per day.

./configure \
  --prefix=/usr \
  --conf-path=/etc/nginx/nginx.conf \
  --http-log-path=/var/log/nginx/access_log \
  --error-log-path=/var/log/nginx/error_log \
  --pid-path=/var/run/nginx.pid \
  --http-client-body-temp-path=/var/tmp/nginx/client \
  --http-proxy-temp-path=/var/tmp/nginx/proxy \
  --http-fastcgi-temp-path=/var/tmp/nginx/fastcgi \
  --with-md5-asm --with-md5=/usr/include \
  --with-sha1-asm \
  --with-sha1=/usr/include \
  --with-http_realip_module \
  --with-http_ssl_module \
  --with-http_perl_module \
  --with-http_stub_status_module

Edit section: Example 4 Example 4

Example on Ubuntu/debian with libgcrypt11-dev, libpcre3-dev and libssl-dev installed (choose EITHER –with-md5 OR –with-sha1, but not both; on debian and ubuntu, they should both point to /usr/lib)

./configure --with-openssl=/usr/lib/ssl/ --with-md5=/usr/lib

An Ubuntu Edgy .deb for version 0.5.2 can be found here: nginx_0.5.2-1_i386.deb .

Edit section: Example 5 Example 5

I use this on RedHat based distros (RHEL, CentOS, Fedora). This is the configuration used for running this wiki.

First, install the dependencies:

yum install gcc openssl-devel pcre-devel zlib-devel

Then run configure:

./configure \
  --prefix=/usr \
  --sbin-path=/usr/sbin/nginx \
  --conf-path=/etc/nginx/nginx.conf \
  --error-log-path=/var/log/nginx/error.log \
  --pid-path=/var/run/nginx/nginx.pid  \
  --lock-path=/var/lock/nginx.lock \
  --user=nginx \
  --group=nginx \
  --with-http_ssl_module \
  --with-http_flv_module \
  --with-http_gzip_static_module \
  --http-log-path=/var/log/nginx/access.log \
  --http-client-body-temp-path=/var/tmp/nginx/client/ \
  --http-proxy-temp-path=/var/tmp/nginx/proxy/ \
  --http-fastcgi-temp-path=/var/tmp/nginx/fcgi/ 

Then finally build and install:

make && make install
Tagged with:
Jun 16

Nginx Server SSL Certificate Installation:

Create a real SSL Certificate

1.Make sure OpenSSL is installed and in your PATH.

2.Create a RSA private key for your Apache server (will be Triple-DES encrypted and PEM formatted):
         $ openssl genrsa -des3 -out server.key 1024
Please backup this server.key file and the pass-phrase you entered in a secure location. You can see the details of this RSA private key by using the command:
         $ openssl rsa -noout -text -in server.key
If necessary, you can also create a decrypted PEM version (not recommended) of this RSA private key with:
         $ openssl rsa -in server.key -out server.key.unsecure

3.Create a Certificate Signing Request (CSR) with the server RSA private key (output will be PEM formatted):
         $ openssl req -new -key server.key -out server.csr
Make sure you enter the FQDN ("Fully Qualified Domain Name") of the server when OpenSSL prompts you for the "CommonName", i.e. when you generate a CSR for a website which will be later accessed via https://www.yourdomain.dom/, enter "www.yourdomain.dom" here. You can see the details of this CSR by using
         $ openssl req -noout -text -in server.csr

4.You now have to send this Certificate Signing Request (CSR) to a Certifying Authority (CA) to be signed. Once the CSR has been signed, you will have a real Certificate, which can be used by Apache. You can have a CSR signed by a commercial CA, or you can create your own CA to sign it.
Commercial CAs usually ask you to post the CSR into a web form, pay for the signing, and then send a signed Certificate, which you can store in a server.crt file. For more information about commercial CAs see the following locations:

  1. Verisign
    http://digitalid.verisign.com/server/apacheNotice.htm
  2. Thawte
    http://www.thawte.com/
  3. CertiSign Certificadora Digital Ltda.
    http://www.certisign.com.br
  4. IKS GmbH
    http://www.iks-jena.de/leistungen/ca/
  5. Uptime Commerce Ltd.
    http://www.uptimecommerce.com
  6. BelSign NV/SA
    http://www.belsign.be

For details on how to create your own CA, and use this to sign a CSR, see below.
Once your CSR has been signed, you can see the details of the Certificate as follows:
$ openssl x509 -noout -text -in yourdomain.crt

Copy the Certificate files to your server:


Copy them(server.key,yourdomain.crt), along with the .key file you generated when you created the CSR, to the directory on your server where you will keep your certificate and key files. Make them readable by root only to increase security.

Edit the Nginx virtual hosts file.

Now open your Nginx virtual host file for the website you are securing. If you need your site to be accessible through both secure (https) and non-secure (http) connections, you will need a server module for each type of connection. Make a copy of the existing non-secure server module and paste it below the original. Then add the lines in bold below:

server {

listen   443;

ssl    on;
ssl_certificate    /etc/ssl/yourdomain.crt;
ssl_certificate_key    /etc/ssl/server.key;

server_name www.yourdomain.com;
access_log /var/log/nginx/nginx.vhost.access.log;
error_log /var/log/nginx/nginx.vhost.error.log;
location / {
	root   /home/www/public_html/your.domain.com/public/;
	index  index.html;
}

}

Adjust the file names to match your certificate files:

  • ssl_certificate should be your primary certificate combined with the intermediate certificate that you made in the previous step (e.g. your_domain_name.crt).
  • ssl_certificate_key should be the key file generated when you created the CSR.
Tagged with:
Jun 13

For securing, you probably want to turn off the version with the directive “server_tokens off”.

server_tokens
syntax: server_tokens on|off
default: server_tokens on
context: http, server, location
Whether to send the Nginx version number in error pages and Server header.

If you want to remove the name of the server completely you need to alter the source code prior to compiling.
Edit /path/to/nginx-0.*/src/http/ngx_http_header_filter_module.c lines 48 and 49:

static char ngx_http_server_string[] = “Server: nginx” CRLF; static char ngx_http_server_full_string[] = “Server: ” NGINX_VER CRLF;

Put in anything you like.
If you want to edit NGINX_VER, it is defined, along with some other relevant constants, in /path/to/nginx-0.*/src/core/nginx.h, lines 11-13.

Refrence:http://wiki.nginx.org/NginxHttpCoreModule#server_tokens
                  http://www.net-square.com/httprint/httprint_paper.html

Tagged with:
Jun 12

Nignx is a very great web server and load banlancer. In my “Nginx Server” category have more about nginx.

If you want to configure load balancer with nginx,you can difine upstream to finish.

deployment overview

real server1
www.domain1.com
192.168.1.11

real server2
www.domain1.com
192.168.1.12

load balancer
balancer.domain1.com
192.168.1.13

master config part:

worker_processes  20;

#error_log  logs/error.log  info;

#pid        logs/nginx.pid;
worker_rlimit_nofile 51200;

events {
      use epoll;
      worker_connections 51200;
}

http {
log_format  www  ‘$remote_addr – $remote_user [$time_local] $request ‘
                        ‘"$status" $body_bytes_sent "$http_referer" ‘
                        ‘"$http_user_agent" "$http_x_forwarded_for" "$gzip_ratio"’;

access_log  logs/access.log  main;
client_header_timeout   10m;
client_body_timeout     10m;
send_timeout            10m;
client_max_body_size     4m;
client_body_buffer_size    256k;

connection_pool_size            256;
client_header_buffer_size       1k;
large_client_header_buffers     4 2k;
request_pool_size               4k;

output_buffers  1 32k;
postpone_output 1460;

tcp_nopush     on;
tcp_nodelay    on;

gzip  on;
gzip_comp_level  3;
gzip_min_length  1100;
gzip_buffers  4 8k;
gzip_proxied any;
gzip_http_version  1.1;
gzip_types  text/plain text/html text/css application/x-javascript text/xml application/xml application/xml+rss text/javascri
pt;

include       mime.types;
default_type  application/octet-stream;
keepalive_timeout 120;

load banlancer config part:

upstream  www.domain1.com  {
        server   192.168.10.11:80;
        server   192.168.10.12:80;
        server   192.168.10.13:80;
}

server
{
        listen  80;
        server_name  www.domain1.com;

        location / {
                 proxy_pass        http://www.domain1.com;
                 proxy_set_header   Host             $host;
                 proxy_set_header   X-Real-IP        $remote_addr;
                 proxy_set_header   X-Forwarded-For  $proxy_add_x_forwarded_for;
        }
}

Tagged with:
Jun 12

Nignx is a very great web server and load banlancer. In my “Nginx Server” category have more about nginx.

If you want to configure multiple mongrel clusters to be used with multiple domains or sub-domains then define multiple upstream blocks and server blocks and add it to the same nginx config file.

master config part:

worker_processes  20;

#error_log  logs/error.log  info;

#pid        logs/nginx.pid;
worker_rlimit_nofile 51200;

events {
      use epoll;
      worker_connections 51200;
}

http {
log_format  www  ‘$remote_addr – $remote_user [$time_local] $request ‘
                        ‘"$status" $body_bytes_sent "$http_referer" ‘
                        ‘"$http_user_agent" "$http_x_forwarded_for" "$gzip_ratio"’;

access_log  logs/access.log  main;
client_header_timeout   10m;
client_body_timeout     10m;
send_timeout            10m;
client_max_body_size     4m;
client_body_buffer_size    256k;

connection_pool_size            256;
client_header_buffer_size       1k;
large_client_header_buffers     4 2k;
request_pool_size               4k;

output_buffers  1 32k;
postpone_output 1460;

tcp_nopush     on;
tcp_nodelay    on;

gzip  on;
gzip_comp_level  3;
gzip_min_length  1100;
gzip_buffers  4 8k;
gzip_proxied any;
gzip_http_version  1.1;
gzip_types  text/plain text/html text/css application/x-javascript text/xml application/xml application/xml+rss text/javascri
pt;

include       mime.types;
default_type  application/octet-stream;
keepalive_timeout 120;

Multiple domains load banlancer config part:

upstream  www.domain1.com  {
        server   192.168.10.11:80;
        server   192.168.10.12:80;
}

server
{
        listen  80;
        server_name  www.domain1.com;

        location / {
                 proxy_pass        http://www.domain1.com;
                 proxy_set_header   Host             $host;
                 proxy_set_header   X-Real-IP        $remote_addr;
                 proxy_set_header   X-Forwarded-For  $proxy_add_x_forwarded_for;
        }

}

upstream  www.domain2.com  {
        server   192.168.10.21:80;
        server   192.168.10.22:80;
}

server
{
        listen  80;
        server_name  www.domain2.com;

        location / {
                 proxy_pass        http://www.domain2.com;
                 proxy_set_header   Host             $host;
                 proxy_set_header   X-Real-IP        $remote_addr;
                 proxy_set_header   X-Forwarded-For  $proxy_add_x_forwarded_for;
        }

        access_log  /usr/local/nginx/logs/domain2.log  www;
}

Tagged with:
Dec 29

Foreword

In my previous article “Using Nginx to enhance the speed of site visits,” introduced Nginx the HTTP server and how to accelerate through its Web site access.  In the actual Web site operators, we often need to understand that the site visit, for example, the number of IP day visit, PV is the number, which URL to visit the largest, most users browser which is what way to know this site and the number of users access to an error and so on, through mastering the information to improve the user experience, thereby improving the quality of the site.  Generally we can visit free of charge through a number of statistics to sites such as Google Analytics or the information.   But the inadequacies of such sites is only an analysis of the page does not include static documents; There may be a lot of regulators do not want to use such tools to expose their own data, all sorts of these factors make regulators want to analyze access logs.  And awstats is enough to meet all these requirements.

Awstats in SourceForge to develop quickly a Perl of WEB-based log analysis tool, a full analysis of the log so that Awstats shows you the following information:

  1. Visits, the number of unique visitors,
  2. Access time and the last visit,
  3. User authentication, the recent certification visit
  4. Weekly peak time (the number of pages, click-through rate per hour and week kilobytes),
  5. Name / country hosts visitors (pages, click-through rate, byte, 269 domains / countries detected, geoip detection),
  6. Host list of recently visited and did not resolve the IP address list
  7. Most have read the entry and exit pages,
  8. , File types,
  9. Site compression tables (mod_gzip or mod_deflate),
  10. Operating system (one for each operating system, the number of pages, click-through rate, byte, 35 OS detected),
  11. Using a browser,
  12. Robot visits (319 robots detected),
  13. Worm attacks (5 worm family),
  14. Search engines, use keyword search to find your address,
  15. HTTP protocol error (the most recent inspection did not find the page),
  16. Other reports based on the personalized URL, link parameters, involving the field of integrated marketing purpose.
  17. Your site by adding “favorite bookmarks.” Views.
  18. Screen size (in the index page of the need to add some HTML tags).
  19. The proportion of browser support: Java, Flash, RealG2 reader, Quicktime reader, WMA reader, PDF reader.
  20. The ratio of load-balancing server cluster report.

Awstats operating environment requires PERL support awstats documents from view, its support for Apache HTTP Server is a very perfect, and when we Nginx replaced after the Web server to run awstats become very troublesome.  First Nginx support Perl itself is relatively weak, and even the official does not recommend the use of; another format in the log there is a need to revised in order to run.

This paper mainly introduces awstats by allowing the outcome of the survey on the log to generate a static page, and then through Nginx statistical output in order to achieve the effect of Nginx access logs, including how to make automatic cutting Nginx log files.

Nginx configure automatic log cutting

With the Apache HTTP Server (hereinafter referred to Apache) The difference is, Apache will log the output of the way through the pipeline re-orientation, and so to automatically log cutting. In the current version of Nginx could not, like Apache, through parameters such as% YY in batches by date to create the log, but nginx process through to send a specific signal, can regenerate nginx log files.  We can implement a Shell script to switch the log, rename or transfer, the specific script is as follows:

# mv  /opt/nginx/logs/access.log /opt/nginx/logs/access_`date +%Y%m%d`.log
# killall –s USR1 nginx

The above script will be saved as a file name content logcron.sh depositors to self-directory, for example, / opt / nginx / sbin / logcron.sh

Crontab allow the use of script in a day 23:59 self-executing, you can do so by the day to create the log.

To install and configure Awstats

Before installing the need to confirm your server Perl environment already in place.

perl –version See the current environment is a Perl version of the command perl-version

We also need to log format Nginx be small changes will not be able to otherwise awstats statistics.

Examples are as follows (bold part):

# vi /opt/nginx/conf/nginx.conf

server {
listen       80;
server_name  localhost;

location ~ ^/web/ {
root   /data/web;
index  index.html;
error_log off;
charset utf-8;
}

log_format  new_log
'$remote_addr - $remote_user [$time_local] $request '
        '"$status" $body_bytes_sent "$http_referer" '
        '"$http_user_agent" "$http_x_forwarded_for"';
        access_log  logs/access.log new_log;
}

 

 

Download the latest version of awstats package, download the address, see the end, the article references. To download the tar package extract to any directory, for example: /usr/local/awstats. And then the implementation of tools directory awstats_configure.pl Configuration Wizard, create a new statistics.

-----> Check for web server install

Enter full config file path of your Web server.
Example: /etc/httpd/httpd.conf
Example: /usr/local/apache2/conf/httpd.conf
Example: c:\Program files\apache group\apache\conf\httpd.conf
Config file path ('none' to skip web server setup):
#> none

Enter

Your web server config file(s) could not be found.
You will need to setup your web server manually to declare AWStats
script as a CGI, if you want to build reports dynamically.
See AWStats setup documentation (file docs/index.html)

-----> Update model config file '/usr/local/awstats/wwwroot/cgi-bin/awstats.model.conf'
  File awstats.model.conf updated.

-----> Need to create a new config file ?
Do you want me to build a new AWStats config/profile
file (required if first install) [y/N] ?
#> y

Enter

-----> Define config file name to create
What is the name of your web site or profile analysis ?
Example: www.mysite.com
Example: demo
Your web site, virtual server or profile name:
#> www.moabc.net
www.moabc.net

Enter

-----> Define config file path
In which directory do you plan to store your config file(s) ?
Default: /etc/awstats
Directory path to store config file(s) (Enter for default):
#>

Enter directly using the default, then there will be the following tips

----> Add update process inside a scheduler
Sorry, configure.pl does not support automatic add to cron yet.
You can do it manually by adding the following command to your cron:
/usr/local/awstats/wwwroot/cgi-bin/awstats.pl -update -config=www.moabc.net
Or if you have several config files and prefer having only one command:
/usr/local/awstats/tools/awstats_updateall.pl now

A SIMPLE config file has been created: /etc/awstats/awstats.www.moabc.net.conf
You should have a look inside to check and change manually main parameters.
You can then manually update your statistics for 'www.moabc.net' with command:
> perl awstats.pl -update -config=www.moabc.net
You can also build static report pages for 'www.moabc.net' with command:
> perl awstats.pl -output=pagetype -config=www.moabc.net

Press ENTER to finish...

Enter the completion of the wizard, then modify configuration www.moabc.net statistics

#vi /etc/awstats/awstats.www.moabc.net.conf
Statistics of the log file to find the path

LogFile=”/var/log/httpd/mylog.log”
Changed
LogFile=”/opt/nginx/logs/access_%YYYY-0%MM-0%DD-0.log

Nginx above the corresponding log cutting procedures generated directory storage structure, attention should be paid to the date Awstats format Nginx with different wording. We are now the order of the implementation of statistics are:

Nginx generated log -> Log Cutting -> Nginx continue to produce the log -> Save cutting logs -> by Awstats Statistics -> generate results

In this article, the statistics Awstats logs, have been cut down that part. Can also reverse the order of a further cut before the end statistics.  But this is easier statistical omission.  Configuration modifications are completed, save quit.  Then we can begin to try manually.

  1. First implementation of the log cutting logcron.sh script to log Nginx cut.
  2. Awstats log and then update the implementation of the beginning of statistical analysis.
# /opt/nginx/sbin/logcron.sh
# /usr/local/awstats/wwwroot/cgi-bin/awstats.pl -update -config=www.moabc.net

Create/Update database for config "/etc/awstats/awstats.www.moabc.net.conf"
        by AWStats version 6.7 (build 1.892)
From data in log file "/opt/nginx/logs/access_20080804.log"...
Phase 1 : First bypass old records, searching new record...
Direct access after last parsed record (after line 450421)
Jumped lines in file: 450421
 Found 450421 already parsed records.
Parsed lines in file: 120
 Found 0 dropped records,
 Found 0 corrupted records,
 Found 0 old records,
 Found 120 new qualified records.

See above shows that the log cutting and running Awstats have been correct. Statistical analysis is completed, the results also Awstats database.  In Apache, you can directly open a Perl program page view statistics.   However, the beginning of this article has already been mentioned, Nginx support of Perl is not good, so we need a change in methods, the use of the tool will awstats the outcome of the survey to generate static documents, concrete steps are as follows:

  • First of all, in the webroot directory, create a folder.
  • Then let Awstats to generate static pages to the directory
# mkdir  /data/webroot/awstats

# /usr/local/awstats/tools/awstats_buildstaticpages.pl -update  \
-config=www.moabc.net -lang=en -dir=/data/admin_web/awstats  \
-awstatsprog=/usr/local/awstats/wwwroot/cgi-bin/awstats.pl

The specific meaning of the order is as follows:

  • /usr/local/awstats/tools/awstats_buildstaticpages.pl  Awstats static page generation tool
  • -update-config = www.moabc.net update the configuration item
  • -lang = en language is English
  • -dir=/data/admin_web/awstats  statistical results output directory
  • -awstatsprog=/usr/local/awstats/wwwroot/cgi-bin/awstats.pl  Awstats log update path.

Next, just nginx.conf in the directory can be configured up.  Examples are as follows: (bold part):

server {
listen       80;
server_name  localhost;

location ~ ^/web/ {
root   /data/web;
index  index.html;
error_log off;
charset utf-8;
}

location ~ ^/awstats/ {
        root   /data/webroot/awstats;
        index  index.html;
        access_log off;
        error_log off;
        charset utf-8;
}

location ~ ^/icon/ {
        root   /usr/local/awstats/wwwroot;
        index  index.html;
        access_log off;
        error_log off;
        charset utf-8;
        }
}

 

Use your browser to view the detailed results of the statistical http://youhostname/awstats/awstats.www.moabc.net.html

At this point, the use of awstats have been able to fully support the Nginx log statistics.

Configured to run automatically Awstats

In order for the entire statistical process log auto-complete, we need to set up crontab scheduled tasks, so that Nginx Log Awstats automatic cutting and running, from time to time to generate results page.

#vi /etc/crontab

11 59 * * * /opt/nginx/sbin/logcron.sh
00 1 * * * /usr/local/awstats/tools/awstats_buildstaticpages.pl  \
-update -config=www.moabc.net -lang=cn -dir=/data/admin_web/awstats \
-awstatsprog=/usr/local/awstats/wwwroot/cgi-bin/awstats.pl  

#00:01  Awstats analysis log

#crontab /etc/crontab

The protection of the log results page

Regulators are generally reluctant to make people aware of their station not the real traffic, so results should Awstats password-protected pages. Nginx using Apache with the same password in encrypted format, where the need to bring their own tools apache used htpasswd.

If you default on this machine equipped with Apache, which you only in its directory run

For example:

#/usr/local/apache2/bin/htpasswd -c admin.pass admin

New password:
Re-type new password:
Adding password for user admin

server {
 	listen       80;
 	server_name  localhost;
 	location ~ ^/web/ {
 	root   /data/web;
 	index  index.html;
 	error_log off;
 	charset utf-8;
 	} 

 	location ~ ^/awstats/ {
        root   /data/admin_web;
        index  index.html;
        access_log off;
        error_log off;
        charset utf-8;
        auth_basic     "admin";
     /opt/ngx/conf/admin.pass;
        }

        location ~ ^/icon/ {
        root   /usr/local/awstats/wwwroot;
        index  index.html;
        access_log off;
        error_log off;
        charset utf-8;
        }
}

Aggregate

Although with the Apache HTTP Server comparison, Nginx function is relatively weak, but we can still use some techniques to circumvent these weaknesses, Nginx designers certainly give full consideration to this issue.  Now a growing number of third-party development in a gradual expansion module Nginx function.  But from the perspective of the application itself, Nginx more inclined to superior performance, rather than function, which in a number of additional functions, we can not too high on the requirements.

Reference resources

  • Nginx English site: http://www.nginx.net
  • Awstats Web site: http://awstats.sourceforge.net
Tagged with:
preload preload preload