Jul 01

Principle:
No optical drive, floppy drive of a server, you want to install Linux. We need an agreement by the PXE network card guide after installing Linux. Process: the machine starts – the NIC boot – Obtain an IP address via DHCP – tftp get through the most basic core file, use the kernel file to start the machine – then you can start the installation configuration, choose to use http, ftp, nfs install remotely access needed packages.

Clearly, the network installation is to be configured with the server. Our server needs to provide the following services:

DHCP,TFTP,NFS(FTP,HTTP)

1.dhcp config

  1. option domain-name "mydomain";
  2. ddns-update-style none;
  3. default-lease-time 600;
  4. max-lease-time 7200;
  5. server-name "bootserver";
  6. subnet 192.168.123.0 netmask 255.255.255.0 {
  7. range 192.168.123.200 192.168.123.201;
  8. deny unknown-clients;
  9. filename "pxelinux.0";
  10. }

filename followed by the tftp directory file, pxelinux.0 is the syslinux package files. Pxelinux.0 may default / usr / lib / syslinux directory must be copied to the tftp directory.

2. tftp config

Edit the file /etc/xinetd.d/tftp (if not, then add the tftp file) (if there xinetd.d, please install the xinetd package)

  1. service tftp
  2. {
  3. disable = no
  4. socket_type = dgram
  5. protocol = udp
  6. wait = yes
  7. user = root
  8. server = /usr/sbin/in.tftpd
  9. server_args = -s /tftpboot
  10. per_source = 11
  11. cps = 100 2
  12. flags = IPv4
  13. }

There will be / tftpboot tftp service is defined as the default directory, you can modify.
After saving restart /etc/init.d/xinetd service, you can open the tftp service.
How to test the success of open tftp?
In the tftp directory, create a file, such as 1.txt.
Tftp service connection in the Shell:
tftp 127.0.0.1
tftp> get 1.txt
If the service successfully opened, you can see a successful download prompt. And in the current directory to find 1.txt file.
Then copy the CD isolinux directory vmlinuz, initrd.img file to the / tftpboot directory.
In the inside / tftpboot folder syslinux.cfg. syslinux.cfg saved pxelinux in two profiles: default, list.

  1. default:
  2. default linux
  3. label linux
  4. kernel vmlinuz
  5. append initrdinitrd=initrd.img devfs=nomount nofb ramdisk_size=9216

You can write a lot of a label, depending on whether you also want this server how many versions of Linux available to the client installation. A version of a label, of course, kernel, and initrd file name can not be repeated.

  1. list:
  2. Choose one of the following Linux distributions for your installation:
  3. Name Distribution Arch. Installation media
  4. CentOS CentOS 4.4 i386 192.168.99.90:/

You can also add multiple lines to select a different release. Choose the time to fill in the contents under Name.

3. copy iso file

Copy the disc to the corresponding directory (ftp, http, nfs), if you are using http using the following command to copy multiple CDs to a directory:

[root@bootserver] # cp -arv /media/cdrom/* /install

4. config nfs

echo ‘/install/ 192.168.123.*(ro.sync)’ >>/etc/exports

        Start nfs Server “/etc/rc.d/init.d/nfs start”

5.  Start Host Press F12 by network boot it.

Tagged with:
Jun 28

Description: myisamchk presents the below error when trying to fix a table with blob columns: miguel@hegel:~/test> myisamchk -r home/miguel/test/*.MYI – recovering (with sort) MyISAM-table ‘/home/miguel/test/table.MYI’ Data records: 43968 – Fixing index 1 Wrong block with wrong total length starting at 76758748 myisamchk: error: Not enough memory for blob at 76758800 MyISAM-table ‘/home/miguel/test/table.MYI’ is not fixed because of errors .

Error Code:

myisamchk: error: myisam_sort_buffer_size is too small
MyISAM-table ‘table.MYI’ is not fixed because of errors

Found link that points at 3255307777713450285 (outside data file) at 5452192
Found link that points at 4049971247778783536 (outside data file) at 5452292
Found link that points at 3255307777713450285 (outside data file) at 5452328
Found link that points at 3978981059712844344 (outside data file) at 5452468
Found link that points at 3978981059712844344 (outside data file) at 5452648
Found link that points at 3978981059712844344 (outside data file) at 5453176
Found link that points at 3978981059712844344 (outside data file) at 5453356
Found block with too small length at 5653824; Skipped
myisamchk: error: Not enough memory for blob at 5656088 (need 1903522412)

While the fix seems obvious (increase the myisam_sort_buffer_size), it’s a bit confusing. The server has both variables, myisam_sort_buffer_size and sort_buffer_size. The first one is the one used for the buffer that is used in “Repair by sort”. Second one is used to buffer a filesort. The ‘myisamchk’ command does NOT have myisam_sort_buffer_size. Myisamchk has ONLY sort_buffer_size, which is used for “Repair by sort”.

The following increases the buffer size that is needed.

myisamchk -o -f tables.MYI –sort_buffer_size=4G

Or increase the memory, if it’s still not sufficient to do the repair. Please your host have enghou memory, and you can copy table to a big memory host and repair it.

Tagged with:
Jun 22

Installing PHP 5.2.x or 5.3.x on RedHat ES5, CentOS 5, etc

To install PHP 5.2.5 (Highest in repository at this time) you can make use of a RPM repository maintained by Remi.

1. download about software

wget http://download.fedora.redhat.com/pub/epel/5/i386/epel-release-5-4.noarch.rpm
wget http://rpms.famillecollet.com/enterprise/remi-release-5.rpm

2. install

rpm -Uvh remi-release-5*.rpm epel-release-5*.rpm

3. update php

yum –enablerepo=remi update php

Tagged with:
Jun 01

If your web server is apache ,you can add follow line:

<Directory /website/attachments>
php_flag engine off
</Directory>

We want disable upload directory php execute privileges on nginx,It’s so simple

location /upload/ {
location ~ .*\.(php)?$
{
deny all;
}
}

limit more directory

location ~* ^/(upload|images)/.*\.(php|php5)$
{
deny all;
}

If your web server is lighthttpd you can

$HTTP["url"] =~ "^/(forumdata|templates|customavatars?)/" {
fastcgi.server = ()
}
Apache:
<Location "/forumdata">
php_admin_flag engine off
Options -ExecCGI
AddType text/plain .html .htm .shtml .php
</Location>

Hope it will help you.

Tagged with:
Mar 21

In my mysql log file appear “Incorrect information in file xxxx.frm” error information.

I try myisamchk fix my talbe

myisamchk -r -q tablename

But the table is bad yet.

frm file only a table struct. So I create a same struct’s new table and copy the new frm file and overwrite bad frm file.

Exciting the bad table is restored.

Tagged with:
Jan 19

Q: apt-get update error:apt error:

E: Dynamic MMap ran out of room.

       how to resolve it.

A: It Is because the default buffer is too small,you can change it. edit /etc/apt/apt.conf

echo “APT::Cache-Limit "125000000";” >> /etc/apt/apt.conf

Tagged with:
Jan 18

Linux Kernel 2.6.20 kernel support for the process of more than IO statistics, can use such tools like iotop each process to monitor the situation on the IO operation, just as with the top real-time view process memory, CPU and so take the case in.  But for Linux kernel version 2.6.20 the following not so lucky, according to Stack Overflow of Replies in this way is given, VPSee Python to write a simple script that is used in linux kernel <2.6.20 the printing process under the IO condition .

Kernel < 2.6.20 Kernel <2.6.20

This idea is very simple script to redirect the results of dmesg to a file and then parse out the process once every 1 second to print the statistics IO read and write, execute this script needs root:

 

#!/usr/bin/python
# Monitoring per-process disk I/O activity
# written by Jason

import sys, os, time, signal, re

class DiskIO:
    def __init__(self, pname=None, pid=None, reads=0, writes=0):
        self.pname = pname
        self.pid = pid
        self.reads = 0
        self.writes = 0

def main():
    argc = len(sys.argv)
    if argc != 1:
        print "usage: ./iotop"
        sys.exit(0)

    if os.getuid() != 0:
        print "must be run as root"
        sys.exit(0)

    signal.signal(signal.SIGINT, signal_handler)
    os.system(‘echo 1 > /proc/sys/vm/block_dump’)
    print "TASK              PID       READ      WRITE"
    while True:
        os.system(‘dmesg -c > /tmp/diskio.log’)
        l = []
        f = open(‘/tmp/diskio.log’, ‘r’)
        line = f.readline()
        while line:
            m = re.match(\
                ‘^(\S+)\((\d+)\): (READ|WRITE) block (\d+) on (\S+)’, line)
            if m != None:
                if not l:
                    l.append(DiskIO(m.group(1), m.group(2)))
                    line = f.readline()
                    continue
                found = False
                for item in l:
                    if item.pid == m.group(2):
                        found = True
                        if m.group(3) == "READ":
                            item.reads = item.reads + 1
                        elif m.group(3) == "WRITE":
                            item.writes = item.writes + 1
                if not found:
                    l.append(DiskIO(m.group(1), m.group(2)))
            line = f.readline()
        time.sleep(1)
        for item in l:
            print "%-10s %10s %10d %10d" % \
                (item.pname, item.pid, item.reads, item.writes)

def signal_handler(signal, frame):
    os.system(‘echo 0 > /proc/sys/vm/block_dump’)
    sys.exit(0)

if __name__=="__main__":
    main()

Kernel> = 2.6.20

If you want to use IO iotop process real-time view the status of activities, then, need to download and upgrade the new kernel (2.6.20 or later).  Compile a new kernel need to open TASK_DELAY_ACCT and TASK_IO_ACCOUNTING options.  Extract the kernel into the configuration interface:

# tar jxvf linux-2.6.30.5.tar.bz2
# mv linux-2.6.30.5 /usr/src/
# cd /usr/src/linux-2.6.30.5

# make menuconfig

Select Kernel hacking -> Collect scheduler debugging info and Collect scheduler statistics, save compile the kernel after kernel:

# Make; make modules; make modules_install; make install

Modify grub, start a new kernel recognized correctly:

# Vi / boot / grub / menu.lst

Out of the new kernel outside, iotop also need to run Python 2.5 or above, so if the current Python 2.4, then you need to download and install the latest Python package.  Compile and install using the source code here:

# Tar jxvf Python-2.6.2.tar.bz2
# cd Python-2.6.2 # Cd Python-2.6.2
# ./configure
#. / Configure # make; make install
# Make; make install

Do not forget to download setuptools:

# Mv setuptools-0.6c9-py2.6.egg.sh setuptools-0.6c9-py2.6.egg # sh setuptools-0.6c9-py2.6.egg # Sh setuptools-0.6c9-py2.6.egg

More

If you want to know more about block_dump information, you can look at this process in real-time Linux IO monitoring the situation .  When using block_dump is best to turn off klogd process.

Tagged with:
Jan 16

Background:

Assuming such a situation, you are a company mysql-DBA, one day all of a sudden the company database was artificially deleted.

Despite the backup, but the service is stopped due to damage caused tens of millions, and now companies need to find out that people who do the delete operation.

However, permission to have database operations a lot of people, how to troubleshoot, Where is the evidence?

Is not that powerless?

mysql itself does not operate the audit function, it is not the means in which case nothing with it?

This article will discuss a simple, ideas for mysql access to the audit.

Keywords: init-connect, binlog, trigger

Overview:

In fact, in itself mysql sql to provide a detailed implementation of records-general log, but it has several drawbacks open

Sql syntax error regardless of whether, if carried out will be recorded, leading to record a lot of useless information, post-screening difficult.

sql concurrency is large, log io record will cause some the impression that the database efficiency.

Rapid expansion of the log file is easy, do not properly handle disk space will cause a certain extent.

This point of view:

Init-connect + binlog using the method of operation of the audit mysql.

As mysql binlog longevity record of all the actual changes to the database sql statement, its execution time, and connection_id But there is no corresponding record connection_id detailed user information.

This article will init-connect, in the initial stage of each connection, the connection of the user record, and connection_id information.

Conduct audits in the latter track, in accordance with the behavior recorded binlog connection-id and the corresponding log records connected with the analysis before, draw final conclusions.

Text:

1. Set init-connect

1.1 create table for sotre user connect log

CREATE DATABASE accesslog;

CREATE TABLE accesslog.accesslog (`id` int(11) primary key auto_increment, `time` timestamp, `localname` varchar(30), `matchname` varchar(30));

1.2 create user for read log infomation

GRANT READ ON accesslog.* to root@localhost identified by ‘password’

1.3 set init-connect

        open my.cnf and at [mysqld] add fllow line

log-bin

init-connect=’insert into accesslog.accesslog values(connection_id(),user(),current_user(),now());’

1.4 restart mysqld

shell>service mysqld restart

2. Record Tracking

 

2.1 thread_id confirmed

Suppose want to know November 25, 2009, more than 9 am when it test.dummy this table who deleted the.  The following statement can be positioning

mysqlbinlog –start-datetime=’2009-11-25 09:00:00′ –stop-datetime=’2009-11-25 09:00:00′ binlog.xxxx | grep ‘dummy’ -B 5

Will get the following results (see thread_id 5):

# at 300777

#091124 16:54:00 server id 10 end_log_pos 301396 Query thread_id=5 exec_time=0 error_code=0

SET TIMESTAMP=1259052840;

drop table test.dummy;

 2.2 the user to confirm

thread_id recognized, find the culprit is just a question of a sql statement.

select * from accesslog.accesslog where conn_id=5 ; select * from accesslog.accesslog where conn_id = 5;

Can be found testuser2 @ localhost dry out.

+——+——————————-+———

| Id | time | localname | matchname |

+——+——————————-+———

| 5 | 2009-11-25 10:57:39 | testuser2@localhost | testuser2@% |

+——+——————————-+———

3. Q & A

Q: using init-connect can affect server performance?

A: In theory, the only connection to the database each time a user to insert a record, will not have a significant impact on the database.  Unless the connection is very high frequency (of course, need to pay attention this time is how to reuse the connection and control, rather than the use of this method is not the problem)

Q: access-log table how to maintain?

A: Because it is a log system, the recommended archive storage engine will help Ecuador compressed data storage.  If the large number of database connection, I suggest a certain time to do a data export, and then clear the table.

Q: What table has other uses?

A: Yes!  access-log table, of course not only for the audit, of course, can also be used for the database connection for data analysis, such as distribution of the daily number of connections, etc., not only can not think of.

Q: there will be missing records?

A: Council, init-connect is not executed when the super user login.  So there will not have access-log record of the database superuser, which is why we do not recommend more than super-user and multi-user reasons.

Tagged with:
Nov 17

Linux, Apache, MySQL and PHP (or Perl) is the basis for many Web applications – from to-do list to blog, then e-commerce site.  WordPress and Pligg are two high-capacity Web site to support a common package.  This architecture referred to as LAMP.  Almost every Linux release contains Apache, MySQL, PHP and Perl, so installing LAMP software is very easy.

Install the misleading simplicity of the software run smoothly on their own, but it is not the case.  Finally, application load will exceed the set comes with back-end server processing power, application performance will be reduced.  LAMP installation requires constant monitoring, tuning and evaluation.

System tuning for different people have different meanings.  This series focuses on components of LAMP (Linux, Apache, MySQL and PHP) tuning.  On tuning the application itself is another complex issue. Back-end server applications, and there is a symbiotic relationship: the server not properly tuned or even the best application would collapse under the load, and tuning with fully, completely avoid the very bad writing application makes the server slow as an ox. Fortunately, the proper system tuning and monitoring can be pointed out that the application in question.

LAMP Architecture

Tuning of any system is the first step in understanding how it works. According to the most simple form, LAMP-based application is written in scripting languages like PHP, they are running on a Linux host part of the Apache Web server running.

PHP applications through the request URL, all the form data and session information has been captured by any access to information from the client to determine what action should be performed.  If necessary, from the MySQL database server (also running on Linux) access to information, some of this information with the Hypertext Markup Language (HTML) templates together, and returns the results to the client. When the user navigate in the application, this process is repeated; when multiple users access the system, this process will be conducted concurrently.  However, the data flow is not unidirectional, because you can use the information from the user to update the database, including session data, statistical data (including voting), and user submitted content (like comments or site update). In addition to dynamic elements, there are static elements, such as images, JavaScript code and Cascading Style Sheets (CSS).

LAMP system in the study at the request stream, take a look at where performance bottlenecks can occur. Provides a number of dynamic information database, the database query response delay will be reflected in the client machine. Web server must be able to quickly execute the script, but also be able to handle multiple concurrent requests. Finally, the underlying operating system must be in good condition to support the application.  Through the network to share files between different servers in other settings may also become a bottleneck.

Performance metrics

Continuous measure of the performance of help in two ways.  First, the performance metrics can help us understand trends, including trends in both good and bad.  As a simple way to look at the Web server central processing unit (CPU) usage, CPU can understand is overloaded.  Also, see the total bandwidth used in the past and to infer future changes that can help determine when the need for network upgrades. The best measure of integration with other measurements and observations to consider.  For example, when users complain about slow application, you can check the disk operation is reaching maximum capacity.

The second use of performance metrics is to determine system performance tuning is a help, or make it worse.  Comparison of change is measured before and after results. However, to make a valid comparison, each should only modify a set of appropriate indicators and then compared to determine the effect of changes. Set a time to modify a reason should be obvious: while the two changes made are likely to affect each other. Selected for comparison of more subtle indicators.

The indicators selected reflect the application user must be able to feel the response. If a revised goal is to reduce the memory footprint of the database, then the cancellation of various buffer will certainly help, but at the expense of query speed and application performance. Therefore, we should select the application response time such indicators, which makes tuning in the right direction, not just for the database memory usage.

Can be measured in many ways the application response time. The easiest way might be to use curl command, as shown in Listing 1.

Listing 1. Using cURL measure the response time of Web site            
$ Curl-o / dev / null-s-w% {time_connect}:% {time_starttransfer}:% {time_total} \
        http://www.test.com

0.081:0.272:0.779

Table 1. curl to use the timer

Timer                           Description

time_connect               TCP connection to the server’s time spent

time_starttransfer        After the request, Web server returns the first byte of the data used in the time

time_total                    The time spent to complete the request

These timers are relative to the transaction start time, or even first in the Domain Name Service (DNS) query.  Therefore, after the request, Web server processes the request and sends back data starting time used 0.272 – 0.081 = 0.191 seconds. Download data from the server the client’s time is used by 0.779 – 0.272 = 0.507 seconds.

By observing the curl data and trends over time, one can see the site’s responsiveness to users.

Of course, Web sites not only by the pages. It also has images, JavaScript code, CSS and cookie to deal with. curl is very suitable for understanding the response time of a single element, but sometimes need to understand the entire page loading speed.

For the Tamper Data Firefox browser extensions (see Resources section for a link) can be recorded in the log for each Web browser sends a request, and displays each request with download time. Method is to use this extension, select Tools> Tamper Data to open the Ongoing requests window. To visit the page loaded, and then the browser will see the status of each request sent and the load time spent for each element.  Figure 1 shows the results of loading developerWorks home page.

Figure 1. DeveloperWorks home page used to load the request of a breakdown

Each line describes one element of the load situation. The data displayed includes the request time, loading time used, size, and results.  Load Duration column lists the time used by the elements themselves, Total Duration column lists all of the time used by child elements.  In Figure 1, the main page loading time is used in 516 milliseconds (ms), but everything loads and displays the time used by the entire page is 5101 ms.

Tamper Data extension there is a useful model, the output of the page to load data into the graphics rendering.  Ongoing requests window, right-click anywhere in the upper part, and select the Graph all. Figure 1 Figure 2 shows a graphical view of data.

Figure 2. DeveloperWorks home page used to load a graphical view of the request

In Figure 2, the duration of each request is displayed as dark blue, and relative to the start time loading the page display.  So, you can see which requests the entire page load slower.

Although the focus is on the page load times and user experience, but do not ignore the core system indicators, such as disk, memory, and network.  There are many utilities to capture this information; which may be the most helpful sar , vmstat and iostat . For more information about these tools, see the Resources section.


Adjust the basic system

The system in Apache, PHP and MySQL tuning components, you should take some time to ensure that components of the underlying Linux operating normally.  It should also be running services on the reduction, only those services required for the operation. This is not only a good security practice, but it will save memory and CPU time.

Kernel tuning some quick measures

Most Linux distributions are defined in the appropriate buffers and other Transmission Control Protocol (TCP) parameters. These parameters can be modified to allocate more memory, thus improving network performance. Set the kernel parameters is through proc interface, that is, through reading and writing /proc values. Fortunately, sysctl can read /etc/sysctl.conf in the value and need to fill under the /proc , so it can more easily manage these parameters.  Listing 2 shows the Internet server Internet server used in some of the more radical of the network settings.

Listing 2. Contains more radical network settings / etc / sysctl.conf    
# Use TCP syncookies when needed
net.ipv4.tcp_syncookies = 1
# Enable TCP window scaling
net.ipv4.tcp_window_scaling: = 1
# Increase TCP max buffer size
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
# Increase Linux autotuning TCP buffer limits
net.ipv4.tcp_rmem = 4096 87380 16777216
net.ipv4.tcp_wmem = 4096 65536 16777216
# Increase number of ports available
net.ipv4.ip_local_port_range = 1024 65000

These settings will be added to /etc/sysctl.conf existing content. The first setting enables TCP SYN cookie.  Sent from the client when a new TCP connection, the SYN packet bit is set, the server connection for the half-open to create an entry, and with a SYN-ACK packet response.  In normal operation, the remote client machine responds with an ACK packet, which will convert the half-open connection to fully open. There is a known as SYN flood (SYN flood) attacks on the network, it can not return the ACK packet, causes the server to run out of memory space, can not handle incoming connections. SYN cookie feature can identify this situation and elegant way to use a reserved space in the queue (for details see reference section). Most systems have this feature enabled by default, but make sure to configure this feature and more reliable.

Enable TCP window stretching to the client to download data at higher speeds.  TCP is not allowed to receive confirmation from the remote end of the case to send multiple packets, the default setting is up to 64 KB, and delay in large remote clients to communicate in this setting may not be enough.  Retractable window will enable more in the first place, thereby increasing the window size.

Four configuration options increase the TCP after sending and receiving buffer. This allows applications to quickly lose its data, so as to request another service. You can also strengthen the remote client when the server is busy sending data.

Finally, a configuration to increase the number of available local port, so that services can increase the maximum number of connections.

The next boot, or the next time you run sysctl -p /etc/sysctl.conf , these settings will take effect.

 Configure disk to improve performance

Disk in the LAMP architecture plays an important role. Static files, templates, and code from disk, composed of the database tables and indexes are from the disk.  Many of the tuning disk (especially for the database) to focus on avoiding disk access, because the disk access latency is quite high.  Therefore, spend some time on the disk hardware optimization is meaningful.

The first step is to ensure that the file system to disable atime logging features.  atime is the time to access files recently, when accessing the file, the underlying file system must record the time stamp. Rarely used because the system administrator atime , disable it can reduce disk access time. This feature is disabled in /etc/fstab to add the fourth column noatime option.  Listing 3 shows an example configuration.

Listing 3. Demonstrates how to enable noatime in fstab example

/dev/VolGroup00/LogVol00 /         ext3    defaults,noatime 1 1
LABEL=/boot             /boot      ext3    defaults,noatime 1 2 
devpts                  /dev/pts   devpts  gid=5,mode=620   0 0
tmpfs                   /dev/shm   tmpfs   defaults         0 0
proc                    /proc      proc    defaults         0 0
sysfs                   /sys       sysfs   defaults         0 0
LABEL=SWAP-hdb2         swap       swap    defaults         0 0
LABEL=SWAP-hda3         swap       swap    defaults         0 0
 
 In Listing 3, only modified the ext3 file system, because the noatime only 
reside on disk file system helpful.  To make the changes to take effect, 
do not need to reboot; just re-mount each file system.  For example, 
to re-mount the root file system, run the mount / -o remount.

Combination of a variety of disk hardware and Linux to access the disk may not be able to detect the best way.  Can use the hdparm command to identify and set the method used to access IDE disks.  hdparm -t /path/to/device speed test, the results of this test can be used as performance benchmarks.  To make the results as accurate as possible, run this command in the system should be idle. Listing 4 shows the hda speed test on the results.

Listing 4. In the /dev/hd speed test performed on

# Hdparm-t / dev / hda
/dev/hda: / Dev / hda:
Timing buffered disk reads:  182 MB in  3.02 seconds =  60.31 MB/sec 
This test shows that in the read data on the disk per second speed is about 60MB.
 

Tuning in to try some disk option, you must pay attention to a problem.  Wrong setting could damage the file system.  Sometimes there will be a warning that this option is not compatible with the hardware; However, sometimes there is no warning message. Therefore, the system into production, you must thoroughly test the settings.  All servers are standard hardware would also help.

Table 2 lists some of the more common options.

Table 2. hdparm common options

Options             Description

-vi –Vi               It supports to set the disk check, and it is set.

-c –C                 hdparm -c 1 /dev/hda enable this setting.

-m –M               uery / set the interrupt for each multi-sector model.  If set greater than zero, set the value is the maximum that can be transmitted for each interrupt number of sectors.

-d 1 -X -D 1-X  Enable Direct Memory Access (DMA) transfer and set the IDE transfer mode. hdparm man page for details in the -X can be set behind the figures. Only in -vi that is not currently the fastest mode of use cases, only the need for this setting.

Unfortunately, for the Fiber Channel and Small Computer Systems Interface (SCSI) system, tuning depends on the specific drive.

Settings must be added to help start the script, such as rc.local .

Network File System tuning

Network File System (NFS) is a network shared disk method.  NFS can help ensure that each host has the same copy of the data and to ensure changes are reflected in all nodes. However, by default, NFS is not configured for high-capacity disk.

Each client should use rsize=32768,wsize=32768,intr,noatime mount remote file system to ensure that:

  • Use of large read / write block (number specifies the maximum block size, in this example is 32KB).
  • When the pending NFS operations can be interrupted.
  • Not continuously updated atime .

These settings can be placed in /etc/fstab in, see Listing 3 .  If you use the automatic mounting device, you should apply these settings in the appropriate /etc/auto.* files.

On the server side, we must ensure that adequate NFS kernel threads to handle all clients. By default, only start a thread, but Red Hat and Fedora system will start 8 threads. For a busy NFS server, this number should be increased, such as 32 or 64.  Can nfsstat -rc command client assessment to see whether the phenomenon of blocking, the command displays the client remote procedure call (RPC) statistics. 

Listing 5 shows a Web server, client statistics.

Listing 5. Display NFS client RPC statistics          
# nfsstat -rc # Nfsstat-rc
Client rpc stats: Client rpc stats:
calls      retrans    authrefrsh calls retrans authrefrsh
1465903813   0          0 146590381300      

The second column retrans is zero, which means that since the last reboot from no need to re-transfer situation.  If this number is relatively large, they should consider increasing the NFS kernel threads.  Setting method is to pass the required number of threads rpc.nfsd , such as rpc.nfsd 128 starts 128 threads. Any time for such settings.  Thread will be needed to start or destroyed. Similarly, this setting should be placed in the startup script, especially in the NFS system to enable scripting.

On the NFS, the last thing to note: if possible, should be avoided NFSv2, because the performance NFSv2 much less than the v3 and v4.  In the modern Linux distributions should not be a problem in that, but you can check on the server nfsstat the output to see if there are any NFSv2 call.

Tagged with:
Nov 12

Nginx from the 0.7.48 release, support for Squid cache similar function. This cache is used as the URL and the relevant combination of Key, with the md5 hash code and saved on the hard drive, so it can support any URL link, also supports 404/301/302 such non-200 status code. Although the official Nginx Web caching service can only status code for the specified URL or set an expiration time, like Squid does not support the PURGE command to manually clear the cache page specified, but Nginx module by a third party, you can clear the cache of the specified URL .
Nginx’s Web caching services are mainly related to the instruction set and fastcgi_cache proxy_cache related instruction set composition, the former agent for the reverse, on the back-end content source server cache, which is mainly used to cache the FastCGI dynamic process. Both features are basically the same.
The latest version of Nginx 0.8.32, proxy_cache and fastcgi_cache been more perfect, with a third party ngx_cache_purge module (used to clear the cache of the specified URL), they can already completely replace the Squid. We have used in the production environment, the proxy_cache Nginx caching more than two months, very stable, speed is not inferior to Squid.
In function, Nginx have already have Squid Web cache acceleration, clear the cache function of the specified URL. In performance, Nginx on the use of multi-core CPU is better than a lot of Squid. In addition, the reverse proxy, load balancing, health checks, back-end server fail, Rewrite rewrite, ease of use, Nginx much stronger than the Squid. This makes a Nginx may also be a "load-balancing server" and "Web cache server" to use.

Install Part:

1.download purge patch for nginx and unzip it
wget http://labs.frickle.com/files/ngx_cache_purge-1.0.tar.gz
tar zxvf ngx_cache_purge-1.0.tar.gz

2.download nginx and unzip nginx
wget http://nginx.org/download/nginx-0.8.32.tar.gz
tar zxvf nginx-0.8.32.tar.gz

3.compile nginx
cd nginx-0.8.32/
./configure –user=www –group=www –add-module=../ngx_cache_purge-1.0 –prefix=/opt/nginx –with-http_stub_status_module –with-http_ssl_module
make && make install
cd /opt/nginx/conf

4.modify config file nginx.conf

user  www www;
worker_processes 8;
error_log  /opt/nginx/logs/nginx_error.log  crit;
pid        /opt/nginx/nginx.pid;
worker_rlimit_nofile 65535;

events
{
  use epoll;
  worker_connections 65535;
}

http
{
  include       mime.types;
  default_type  application/octet-stream;

  #charset  utf-8;

  server_names_hash_bucket_size 128;
  client_header_buffer_size 32k;
  large_client_header_buffers 4 32k;
  client_max_body_size 300m;
  sendfile on;
  tcp_nopush     on;
  keepalive_timeout 60;
  tcp_nodelay on;

  client_body_buffer_size  512k;
  proxy_connect_timeout    5;
  proxy_read_timeout       60;
  proxy_send_timeout       5;
  proxy_buffer_size        16k;
  proxy_buffers            4 64k;
  proxy_busy_buffers_size 128k;
  proxy_temp_file_write_size 128k;

#gzip on;
#gzip_min_length  1k;
#gzip_buffers     4 16k;
#gzip_http_version 1.1;
#gzip_comp_level 2;
#gzip_types       text/plain application/x-javascript text/css application/xml;
#gzip_vary on;

proxy_temp_path   /data/proxy_temp_dir;
proxy_cache_path  /data/proxy_cache_dir  levels=1:2   keys_zone=cache_one:500m inactive=1d max_size=20g;

upstream backend_server {
  server   192.168.1.38:80 weight=1 max_fails=2 fail_timeout=30s;
}

server
{
  listen       80;
  server_name  www.test.com *.test.com;
  index index.html index.htm index.php;

  location /
  {
       proxy_next_upstream http_502 http_504 error timeout invalid_header;
       proxy_cache cache_one;
       proxy_cache_valid  200 304 12h;

       proxy_cache_key $host$uri$is_args$args;
       proxy_set_header Host  $host;
       proxy_set_header X-Forwarded-For  $remote_addr;
       proxy_pass http://backend_server;
       expires      1d;
  }

  location ~ /purge(/.*)
  {
   allow            127.0.0.1;
   allow            192.168.1.253;
   deny             all;
   proxy_cache_purge    cache_one   $host$1$is_args$args;
    }

    location ~ .*\.(php|jsp|cgi)?$
    {
         proxy_set_header Host  $host;
         proxy_set_header X-Forwarded-For  $remote_addr;
         proxy_pass http://www.test.com;
    }

    access_log  off;
  }
}

5. Nginx start script nginx-start.sh
#!/bin/sh
ulimit -SHn 65535
/opt/nginx/sbin/nginx

chmod +x nginx-start.sh
./nginx-start.sh

6.Clean special url

you can visit page to delete

upload/thumb/20101101/201011011126134751_120_90.jpg  file.
http://www.test.com/purge/upload/thumb/20101101/201011011126134751_120_90.jpg

Return Result

Successful purge

Key : www.test.com/upload/thumb/20101101/201011011126134751_120_90.jpg
Path: /data/proxy_cache_dir/0/98/d0a52447df34c0d5abe1cf34b4bf0980


nginx/0.8.32


Tagged with:
preload preload preload