Jun 05

deployment overview

web server
site1.yourdomain.com
192.168.1.35

web server
siste2.yourdomain.com
192.168.1.36

data server
db1.yourdomain.com
192.168.1.37

load balancer
balance-1.yourdomain.com
192.168.1.45

install and enable apache and proxy_balancer

1.create a dedicated server for load balancing. install apache2 and then install mod proxy_balancer and proxy_http with dependencies.

2.enable mod_proxy in httpd.conf. note that i’m leaving ProxyRequests off since we’re only using the ProxyPass and ProxyPassReverse directives. this keeps the server secure from spammers trying to use your proxy to send email.

<IfModule mod_proxy.c>
        ProxyRequests Off

        <Proxy *>
                AddDefaultCharset off
                Order deny,allow
                Allow from all
                #Allow from .example.com
        </Proxy>

        ProxyVia On
</IfModule>

configure mod_proxy and mod_proxy_balancer

mod_proxy and mod_proxy balancer serve as a very functional load balancer. however mod_proxy_balancer makes slightly unfortunate assumptions about the format of the cookie that you’ll use for sticky session handling. one way to work around this is to create your own session cookie (very easy with apache). the examples below describe how to do this

first create a virtual host or use the default  and add this configuration to it:

<Location /balancer-manager>
SetHandler balancer-manager
Order Deny,Allow
Deny from all
Allow from 192.168
</Location>
<Proxy balancer://mycluster>
  # cluster member 1
  BalancerMember
http://site1.yourdomain.com:80 route=lb1
  # cluster member 2
  BalancerMember
http://site2.yourdomain.com:80 route=lb2
</Proxy>
ProxyPass /balancer-manager !
ProxyPass / balancer://mycluster/ lbmethod=byrequests stickysession=BALANCEID
ProxyPassReverse /
http://site1.yourdomain.com/
ProxyPassReverse / http://site2.yourdomain.com/

Note:

  • i’m allowing access to the balancer manager from any IP matching 192.168.*.*
  • i’m load balancing between 2 servers (site1.yourdomain.com, site2.yourdomain.com) on port 80
  • i’m defining two routes for these servers called lb1 and lb2
  • i’m excluding (!) the balancer-manager directory fro the ProxyPass to allow access to the manager ui on the load balancing server
  • i’m expecting a cookie called BALANCEID to be available to manage sticky sessions
  • this is a simplistic load balancing configuration. apache has many options to control timeouts, server loading, failover etc. too much to cover but read more in the apache documentation

    configure the web servers to write a session cookie

    on each of the web servers, add this code to your vhost configuration:

    RewriteEngine On
    RewriteRule .* - [CO=BALANCEID:balancer.lb1:.yourdomain.com]

    making sure to specify the correct route e.g. lb1 on site1.yourdomain.com etc.

    you also probably want to setup your cookie domain properly in drupal, i.e. modify drupal/sites/default/settings.php as follows:

    # $cookie_domain = 'example.com';
    $cookie_domain = 'yourdomain.com';

    important urls

    useful urls for testing are:

    References:

  • apache’s mod_proxy_balancer documentation
  • apache’s mod_proxy documentation

     

    About Load Banlance topic you can read my old post:

  • Tagged with:
    May 19

    Varnish is a state-of-the-art, high-performance HTTP accelerator. It uses the advanced features in Linux 2.6, FreeBSD 6/7 and Solaris 10 to achieve its high performance.

    Some of the features include

    • A modern design
    • VCL – a very flexible configuration language
    • Load balance with health checking of backends
    • Partial support for ESI
    • URL rewriting
    • Graceful handling of "dead" backends

    INSTALL:

    1.download varnish-2.0.4.tar.gz
    If you want the last version,you can hit here.

    2.unzip varnish
    tar zxvf varnish-2.0.4.tar.gz

    3.compile varnish
    cd varnish-2.0.4
    ./configure –prefix=/usr/local/varnish –enable-debugging-symbols –enable-developer-warnings –enable-dependency-tracking –enable-tests –enable-diagnostics
    make
    make install

    4.create user:
    adduser -s /sbin/nologin www

    5.create cache directory &  log file
    mkdir -p /var/vcache && chown www.www /var/vcache/
    mkdir -p /usr/local/varnish/var/logs && chown www.www /usr/local/varnish/var/logs

    6.create config file:
    vcl.conf
    backend photo1 {
           .host = "10.0.5.33";
           .port = "80";
            .probe = {
                    .url = "/guide.php";
                    .timeout = 50ms;
                    .interval = 5s;
                    .window = 10;
                    .threshold = 8;
            }
    }
    backend photo2 {
           .host = "10.0.7.33";
           .port = "80";
            .probe = {
                    .url = "/guide.php";
                   .timeout = 50ms;
                    .interval = 5s;
                    .window = 10;
                    .threshold = 8;
            }
    }
    director photo_director round-robin {
        { .backend = photo1; }
        { .backend = photo2; }
    }
    acl purge {
           "localhost";
           "127.0.0.1";
           "10.0.0.0"/16;
    }

    sub vcl_recv {
           if (req.request == "PURGE") {
                   if (client.ip ~ purge) {
                           purge_url(req.url);
                    }
                    else{
                            error 200 "Purged";
                    }
           }
           if (req.url ~ "\.(gif|jpg|jpeg|bmp)$") {
                    unset req.http.cookie;
                    unset req.http.authenticate;
                    set req.backend = photo_director;
           }
    }

    sub vcl_hit {
           if (req.request == "PURGE") {
                   set obj.ttl = 0s;
                   error 200 "Purged.";
           }
            deliver;
    }

    sub vcl_hash {
        set req.hash += req.url;
        set req.hash += req.http.host;
        set req.hash += req.http.cookie;
        hash;
    }

    sub vcl_miss {
           if (req.request == "PURGE") {
                   error 404 "Not in cache.";
           }
    }

    sub vcl_fetch {
        if (!obj.cacheable) {
            pass;
        }
        if (obj.http.Pragma ~ "no-cache" || obj.http.Cache-Control ~ "no-cache" || obj.http.Cache-Control ~ "private") {
            pass;
        }
        deliver;
    }

    7.Optimize the kernel parameters:
    net.ipv4.ip_local_port_range = 1024 65536
    net.core.rmem_max=16777216
    net.core.wmem_max=16777216
    net.ipv4.tcp_rmem=4096 87380 16777216
    net.ipv4.tcp_wmem=4096 65536 16777216
    net.ipv4.tcp_fin_timeout = 3
    net.ipv4.tcp_tw_recycle = 1
    net.core.netdev_max_backlog = 30000
    net.ipv4.tcp_no_metrics_save=1
    net.core.somaxconn = 262144
    net.ipv4.tcp_syncookies = 0
    net.ipv4.tcp_max_orphans = 262144
    net.ipv4.tcp_max_syn_backlog = 262144
    net.ipv4.tcp_synack_retries = 2
    net.ipv4.tcp_syn_retries = 2

    8.Start varnish:
    #!/bin/sh
    ulimit -SHn 51200
    /usr/local/varnish/sbin/varnishd -n /var/vcache -f /usr/local/varnish/vcl-php.conf -a 0.0.0.0:80 -s malloc -g www -u www -w 500,10000,15 -T 127.0.0.1:3500 -p client_http11=on -plisten_depth=4096 -p lru_interval=3600  -h classic,500009 -p obj_workspace=2048
    /usr/local/varnish/bin/varnishncsa -n /var/vcache -w /usr/local/varnish/var/logs/varnish.log &

    More Detail

    Tagged with:
    May 14

    HAProxy is a TCP/HTTP reverse proxy which is particularly suited for high availability environments. Indeed, it can : – route HTTP requests depending on statically assigned cookies ; – spread the load among several servers while assuring server persistence through the use of HTTP cookies ; – switch to backup servers in the event a main one fails ; – accept connections to special ports dedicated to service monitoring ; – stop accepting connections without breaking existing ones ; – add/modify/delete HTTP headers both ways ; – block requests matching a particular pattern ;

    Install:
    1.download haproxy.
    wget http://haproxy.1wt.eu/download/1.3/src/haproxy-1.3.18.tar.gz
    You can get the last version from the website

    2.unzip haproxy
    tar zxvf  haproxy-1.3.18.tar.gz

    3.compile
    make TARGET=linux26
    More detail you can look for the file README of the soure files.
    mv haproxy-1.3.18 /usr/local/haproxy

    4.create config file
    vi /usr/local/haproxy/proxy.cfg
    global
            uid 99
            gid 99
            maxconn 48000
            chroot  /usr/local/haproxy
            daemon
            quiet
            nbproc  2
            #pidfile /var/run/haproxy-private.pid

    defaults
            log     global
            mode    http
            option  httplog
            option  dontlognull
            log 127.0.0.1 local3
            retries 3
            option  redispatch
            maxconn 2000
            contimeout      5000
            clitimeout      50000
            srvtimeout      50000

    listen http-proxy :80
            mode http
            stats uri /haproxy-stats
            stats realm Haproxy\ statistics
            stats auth monitor:StatusM5776
            balance roundrobin
            #balance source
            cookie PHPSESSID prefix
            option httpclose
            option forwardfor
            option httpchk HEAD /adima/logo.gif HTTP/1.0
            appsession PHPSESSID len 32 timeout 86400000
            server ht1 192.168.10.1:80 cookie ht1 check inter 3000 rise 2 fall 3
            server ht2 192.168.10.2:80 cookie ht2 check inter 3000 rise 2 fall 3
            server ht3 192.168.10.3:80 cookie ht3 check inter 3000 rise 2 fall 3

    5.start haproxy shell

    #!/bin/sh
    # chkconfig: 2345 80 32
    # description: haproxy is a Web Agent, which is the program \

    ulimit -SHn 51200
    MPRO="/usr/local/haproxy/haproxy"
    DCFG="/usr/local/haproxy/proxy.cfg"

    [ -x ${MPRO} ] || exit 0

    RETVAL=0

    start () {
            echo -n "Starting haproxy: "
            ${MPRO} -f $DCFG
            RETVAL=$?
           [ $RETVAL -eq 0 ] && touch /var/lock/subsys/haproxy
            echo
           return $RETVAL
    }

    stop() {
           # Stop daemons.
           echo -n "Shutting down haproxy: "
           killall haproxy
           RETVAL=$?
           [ $RETVAL -eq 0 ] && rm -f /var/lock/subsys/haproxy
           echo
           return $RETVAL
    }

    restart() {
            stop
            sleep 1
            start
    }

    case "$1" in
    start)

           start
           ;;
    stop)
           stop
           ;;
    restart)
           restart
           ;;
    *)
           echo "Usage: haproxy {start|stop|restart}"
           exit 1
    esac

    exit $?

    Tagged with:
    preload preload preload