Nov 15

MMM (Multi-Master Replication Manager for MySQL) is a set of flexible scripts to perform monitoring/failover and management of MySQL master-master replication configurations (with only one node writable at any time).

The toolset also has the ability to read balance standard master/slave configurations with any number of slaves, so you can use it to move virtual IP addresses around a group of servers depending on whether they are behind in replication.

The current version of this software is stable, but the authors would appreciate any comments, suggestions, bug reports about this version to make it even better. Current version 2.0 development is led by Pascal Hofmann. If you require support, advice or assistance with deployment, please contact Percona or Open Query.

Tagged with:
Jun 28

Description: myisamchk presents the below error when trying to fix a table with blob columns: miguel@hegel:~/test> myisamchk -r home/miguel/test/*.MYI – recovering (with sort) MyISAM-table ‘/home/miguel/test/table.MYI’ Data records: 43968 – Fixing index 1 Wrong block with wrong total length starting at 76758748 myisamchk: error: Not enough memory for blob at 76758800 MyISAM-table ‘/home/miguel/test/table.MYI’ is not fixed because of errors .

Error Code:

myisamchk: error: myisam_sort_buffer_size is too small
MyISAM-table ‘table.MYI’ is not fixed because of errors

Found link that points at 3255307777713450285 (outside data file) at 5452192
Found link that points at 4049971247778783536 (outside data file) at 5452292
Found link that points at 3255307777713450285 (outside data file) at 5452328
Found link that points at 3978981059712844344 (outside data file) at 5452468
Found link that points at 3978981059712844344 (outside data file) at 5452648
Found link that points at 3978981059712844344 (outside data file) at 5453176
Found link that points at 3978981059712844344 (outside data file) at 5453356
Found block with too small length at 5653824; Skipped
myisamchk: error: Not enough memory for blob at 5656088 (need 1903522412)

While the fix seems obvious (increase the myisam_sort_buffer_size), it’s a bit confusing. The server has both variables, myisam_sort_buffer_size and sort_buffer_size. The first one is the one used for the buffer that is used in “Repair by sort”. Second one is used to buffer a filesort. The ‘myisamchk’ command does NOT have myisam_sort_buffer_size. Myisamchk has ONLY sort_buffer_size, which is used for “Repair by sort”.

The following increases the buffer size that is needed.

myisamchk -o -f tables.MYI –sort_buffer_size=4G

Or increase the memory, if it’s still not sufficient to do the repair. Please your host have enghou memory, and you can copy table to a big memory host and repair it.

Tagged with:
Jan 16

Background:

Assuming such a situation, you are a company mysql-DBA, one day all of a sudden the company database was artificially deleted.

Despite the backup, but the service is stopped due to damage caused tens of millions, and now companies need to find out that people who do the delete operation.

However, permission to have database operations a lot of people, how to troubleshoot, Where is the evidence?

Is not that powerless?

mysql itself does not operate the audit function, it is not the means in which case nothing with it?

This article will discuss a simple, ideas for mysql access to the audit.

Keywords: init-connect, binlog, trigger

Overview:

In fact, in itself mysql sql to provide a detailed implementation of records-general log, but it has several drawbacks open

Sql syntax error regardless of whether, if carried out will be recorded, leading to record a lot of useless information, post-screening difficult.

sql concurrency is large, log io record will cause some the impression that the database efficiency.

Rapid expansion of the log file is easy, do not properly handle disk space will cause a certain extent.

This point of view:

Init-connect + binlog using the method of operation of the audit mysql.

As mysql binlog longevity record of all the actual changes to the database sql statement, its execution time, and connection_id But there is no corresponding record connection_id detailed user information.

This article will init-connect, in the initial stage of each connection, the connection of the user record, and connection_id information.

Conduct audits in the latter track, in accordance with the behavior recorded binlog connection-id and the corresponding log records connected with the analysis before, draw final conclusions.

Text:

1. Set init-connect

1.1 create table for sotre user connect log

CREATE DATABASE accesslog;

CREATE TABLE accesslog.accesslog (`id` int(11) primary key auto_increment, `time` timestamp, `localname` varchar(30), `matchname` varchar(30));

1.2 create user for read log infomation

GRANT READ ON accesslog.* to root@localhost identified by ‘password’

1.3 set init-connect

        open my.cnf and at [mysqld] add fllow line

log-bin

init-connect=’insert into accesslog.accesslog values(connection_id(),user(),current_user(),now());’

1.4 restart mysqld

shell>service mysqld restart

2. Record Tracking

 

2.1 thread_id confirmed

Suppose want to know November 25, 2009, more than 9 am when it test.dummy this table who deleted the.  The following statement can be positioning

mysqlbinlog –start-datetime=’2009-11-25 09:00:00′ –stop-datetime=’2009-11-25 09:00:00′ binlog.xxxx | grep ‘dummy’ -B 5

Will get the following results (see thread_id 5):

# at 300777

#091124 16:54:00 server id 10 end_log_pos 301396 Query thread_id=5 exec_time=0 error_code=0

SET TIMESTAMP=1259052840;

drop table test.dummy;

 2.2 the user to confirm

thread_id recognized, find the culprit is just a question of a sql statement.

select * from accesslog.accesslog where conn_id=5 ; select * from accesslog.accesslog where conn_id = 5;

Can be found testuser2 @ localhost dry out.

+——+——————————-+———

| Id | time | localname | matchname |

+——+——————————-+———

| 5 | 2009-11-25 10:57:39 | testuser2@localhost | testuser2@% |

+——+——————————-+———

3. Q & A

Q: using init-connect can affect server performance?

A: In theory, the only connection to the database each time a user to insert a record, will not have a significant impact on the database.  Unless the connection is very high frequency (of course, need to pay attention this time is how to reuse the connection and control, rather than the use of this method is not the problem)

Q: access-log table how to maintain?

A: Because it is a log system, the recommended archive storage engine will help Ecuador compressed data storage.  If the large number of database connection, I suggest a certain time to do a data export, and then clear the table.

Q: What table has other uses?

A: Yes!  access-log table, of course not only for the audit, of course, can also be used for the database connection for data analysis, such as distribution of the daily number of connections, etc., not only can not think of.

Q: there will be missing records?

A: Council, init-connect is not executed when the super user login.  So there will not have access-log record of the database superuser, which is why we do not recommend more than super-user and multi-user reasons.

Tagged with:
Mar 13

We recently came across a problem with Cacti and the MySQL counters. For those of you who don’t know how to integrate MySQL statistics into Cacti have a look at this: http://code.google.com/p/mysql-cacti-templates/. These templates are a great way to gain some insight into how your MySQL database servers perform. The templates are actually PHP pages that query the databases through a variety of commands like SHOW STATUS and SHOW ENGINE INNODB STATUS.

The issue that we encountered was that some statistics like the InnoDB buffer pool activity were not displaying anything for one server. Other servers were displaying it just fine and other statistics for that server were also fine.

Among other things the SHOW ENGINE INNODB STATUS command shows deadlock information pertaining to the last deadlock that the InnoDB engine encountered. In some cases this information will be quite extensive and this causes a problem. The output of this command is one giant text field with a limit of 64KB. If the deadlock information is very large other information will get cut off which means certain statistics are lost. The easy fix for this is to restart the database server but in case this is not an option you can always use the innotop utility to wipe the deadlock information by causing a small deadlock.

Tagged with:
Mar 09

Here’s an interesting one, what if you have a MySQL replication setup and the slave stops replicating with a syntax error? The slave should be executing the exact same commands as the master, right? Well, as it turns out, yes and no. There is a bug in MySQL that has been fixed in 5.0.56 according to the bug report. It’s a long story and it’s worth the read but what happens is that a timeout in the network connection between the master and the slave can cause the master to resend part of packet that it sent before. The slave handled the previous packet correctly so it’s not expecting a resend and as a result it starts writing some garbage to the relay log (which is where it stored the statements it will execute). The SQL command gets mangled in the process and when the slave tries to execute it, voila, a syntax error.

To fix this you can use the CHANGE MASTER command to set the slave to the master bin log file and position that shows up in the SHOW SLAVE STATUS output. Make sure you use the Relay_Master_Log_File and Exec_Master_Log_Pos fields since they indicate what position in the master binlog the slave actually thought it was executing. Keep in mind that corruption and its effects are hard to predict. It will definitely be useful to compare the master and slave afterward using the MaatKit tools.

As some more background, the server log will be probably show and error like this to indicate there was a network error:
[ERROR] Error reading packet from server: Lost connection to MySQL server during query (server_errno=2013)

And finally, if you do read the entire bug thread you will notice that the original developer of MySQL also has an opinion on this.

Tagged with:
Mar 03

If you want to grant remote access privileges to a DB on your server,Please look fllow.

First login trough SSH on your server and get access to you mysql

$mysqladmin -u xxx password  xxx

Note: the db user and db passwd are your database username and database password

GRANT ALL PRIVILEGES ON db_base.* TO db_user @’%’ IDENTIFIED BY ‘db_passwd’;

You can also grant accesss to a specific IP adress

GRANT ALL PRIVILEGES ON failserv_example.* TO failserver@’xx.xx.xx.xx’ IDENTIFIED BY ‘db_passwd’;

(where x is your own remote IP)

Refresh privileges,After that to activate your setting type

FLUSH PRIVILEGES;

And exit your mysql

$mysql>quit

Tagged with:
Dec 17

Description:
Currently the number of fd’s on windows is limited to 2048.
This is bad for performance, for many reasons, mostly windows servers
are stuck with a tiny table_cache or low number of concurrent connections,
compared with linux running on the same hardware.

Error Log:
Could not increase number of max_open_files to more than 2048 (request: 3082)

Othre Error:

ERROR 1135: Can’t create a new thread (errno 11). If you are not out of available memory, you can consult the manual for a possible OS-dependent bug

How to repeat:
Run the server

mysqld-nt.exe –table_cache=1000 –max_connections=500 –open_files-limit=3000 –console

061122 17:04:27 [Warning] Could not increase number of max_open_files to more than 2048
(request: 2510)
061122 17:04:28  InnoDB: Started; log sequence number 0 43655
061122 17:04:28 [Note] mysqld-nt: ready for connections.
Version: ’5.0.30-enterprise-gpl-nt’  socket: ”  port: 3306  MySQL Enterprise Server
(GPL)

More:http://bugs.mysql.com/bug.php?id=24509

Solution:

1.Update Your Mysql Server to Mysql5.5,It has been fix is released in 5.5.

2.Replacement of the operating system,Linux have not the problem.

Tagged with:
Dec 02

Mysql Full Backup Script, Hope it can help you.

#!/bin/sh
###################################################################
# Name:Mysql_Full_Backup.sh
# PS:MySQL DataBase Full Backup.
# Write by:Jason
# Last Modify:2009-11-20
###################################################################

# Define Variable Please Modify By Fact
# Define Script Directory
scriptsDir=/home/Script

# Define Database Directory
mysqlDir=/srv/mysql

# Define Database User & Name
user=bkuser
userPWD=pwd

# Define Backup Directory
dataBackupDir=/home/mysqlbackup

# Define Email Content
eMailFile=$dataBackupDir/email.txt

# Define Email Address
eMail=xxxx@mail.com

# Define Backup Log File.
logFile=$dataBackupDir/mysqlbackup.log
DATE=`date -I`

echo "" > $eMailFile
echo $(date +"%y-%m-%d %H:%M:%S") >> $eMailFile
cd $dataBackupDir

# Define Backup Filename.
dumpFile=database_$DATE.sql
GZDumpFile=database_$DATE.sql.tar.gz

# Backup Database By mysqldump
$mysqlDir/bin/mysqldump -u$user -p$userPWD \
–opt –default-character-set=gbk –extended-insert=false \
–triggers -R –hex-blob –all-databases \
–flush-logs –delete-master-logs \
–lock-all-tables > $dumpFile

# Compress Backup File

if [[ $? == 0 ]]; then
  tar czf $GZDumpFile $dumpFile >> $eMailFile 2>&1
  echo "BackupFileName:$GZDumpFile" >> $eMailFile
  echo "DataBase Backup Success!" >> $eMailFile

  rm -f $dumpFile

# Delete daily backup files.
#  cd $dataBackupDir/daily
#  rm -f *

# Delete old backup files(mtime>2).
#  $scriptsDir/rmBackup.sh

# Move Backup Files To Backup Server.
$scriptsDir/Rsync_Backup.sh

  if (( !$? )); then
    echo "Move Backup Files To Backup Server Success!" >> $eMailFile
    else
    echo "Move Backup Files To Backup Server Fail!" >> $eMailFile
  fi
else
  echo "DataBase Backup Fail!" >> $emailFile
fi

# Write Log File…
#echo "——————————————————–" >> $logFile

cat $eMailFile >> $logFile

# Notify ADMIN by Email.
#cat $eMailFile | mail -s "MySQL Backup" $eMail

Tagged with:
Aug 06

Of course, the data you put into MySQL can be any language you want, but many people around the world who do not speak English as a first language use MySQL. MySQL AB, the company now responsible for MySQL, is based in Sweden, and most of the primary developers are Scandinavian. So, it comes as no surprise that MySQL distributions come with support for other languages. The following languages are currently supported, and more are likely to be added: Czech, Danish, Dutch, English (the default), Estonian, French, German, Greek, Hungarian, Italian, Korean, Norwegian, Norwegian-ny, Polish, Portuguese, Romanian, Russian, Slovak, Spanish, and Swedish.

Displaying Error Messages in Another Language

Starting MySQL so that it displays error messages in one of these languages is as easy as using
the –language or -L options at startup. To do so in the config file, simply add a line such as the following:

language=french

You can also edit the error messages yourself (perhaps you want your database to have that personal touch) or contribute your own set in another language, and give something back to the MySQL community. To change the error messages, simply edit the errmsg.txt file in the appropriate language directory (usually /share/language_name from the MySQL base directory), run the cmp_error utility, and restart the server. For example:

% cp errmsg.txt errmsg.bak
% vi errmsg.txt

Here I edited the error message that read as follows:

"No Database Selected",

to read as follows instead:

"Haven't you forgotten something - No Database Selected",

and then saved it:

"errmsg.txt" 229 lines, 12060 characters written
% comp_err errmsg.txt errmsg.sys
Found 226 messages in language file errmsg.sys

Then restart the server, and the new error messages will take effect:

% mysqladmin shutdown
% /etc/rc.d/init.d/mysql start
% mysql -uroot -pg00r002b
mysql> SELECT * FROM a;
ERROR 1046: Haven't you forgotten something - No Database Selected

You’ll have to repeat the changes if you upgrade to a newer version of MySQL.

Using a Different Character Set

By default, MySQL uses the Latin1 (ISO-8859-1) character set. The character set determines what characters can be used, as well as the sorting order for queries. You can change the character set by changing the value of the –default-character-set option when you start the server. The available character sets currently include the following:

latin1

dos

estonia

big5

german1

hungarian

czech

hp8

koi8_ukr

euc_kr

koi8_ru

win1251ukr

gb2312

latin2

greek

gbk

swe7

win1250

latin1_de

usa7

croat

sjis

cp1251

cp1257

tis620

danish

latin5

ujis

hebrew

 

dec8

win1251

 

You can see what character sets are available in your distribution by looking at the value of the character_sets variable.

When you change a character set, you’ll need to rebuild your indexes to ensure they sort according to the rules of the new character set.

By default, MySQL is compiled with –with-extra-charsets=complex, which makes the other character sets available if necessary. If you are compiling MySQL yourself, and you know you are never going to need other character sets, you can use the –with-extra-charsets=none option.

Adding Your Own Character Set

You can add your own character set as well. If it is a simple character set and does not need multibyte character support or string collating routines for sorting, adding it is easy. It becomes more complex if these extras are required. To add a character set, perform the following steps:

  1. Add the new character set to the sql/share/charsets/Index file, and give it a unique ID. The path may differ on some distributions, but it’ll always be the Index file. Here, you can call the new character set martian, with an ID of 31:

    # sql/share/charsets/Index
    #
    # This file lists all of the available character sets.
    
    big5               1
    czech              2
    dec8               3
    dos                4
    german1            5
    hp8                6
    koi8_ru            7
    latin1             8
    latin2             9
    swe7              10
    usa7              11
    ujis              12
    sjis              13
    cp1251            14
    danish            15
    hebrew            16
    # The win1251 character set is deprecated.  Please use cp1251 instead.
    win1251           17
    tis620            18
    euc_kr            19
    estonia           20
    hungarian         21
    koi8_ukr          22
    win1251ukr        23
    gb2312            24
    greek             25
    win1250           26
    croat             27
    gbk               28
    cp1257            29
    latin5            30
    martian           31
  2. Create the .conf and place it in the directory, for example, sql/share/charsets/martian.conf. Use one of the existing .conf files as a starting point for this.

    In the .conf file, lines beginning with a # are comments, words are separated by any amount of whitespace, and every word must be in hexadecimal format. There are four arrays. In order, they are ctype (containing 257 elements), to_lower and to_upper (each containing 256 elements), and sort_order (also containing 256 elements). The following is a sample .conf file (this is the standard latin1.conf):

    # Configuration file for the latin1 character set
    
    # ctype array (must have 257 elements)
      00
      20  20  20  20  20  20  20  20  20  28  28  28  28  28  20  20
      20  20  20  20  20  20  20  20  20  20  20  20  20  20  20  20
      48  10  10  10  10  10  10  10  10  10  10  10  10  10  10  10
      84  84  84  84  84  84  84  84  84  84  10  10  10  10  10  10
      10  81  81  81  81  81  81  01  01  01  01  01  01  01  01  01
      01  01  01  01  01  01  01  01  01  01  01  10  10  10  10  10
      10  82  82  82  82  82  82  02  02  02  02  02  02  02  02  02
      02  02  02  02  02  02  02  02  02  02  02  10  10  10  10  20
      00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00
      00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00
      48  10  10  10  10  10  10  10  10  10  10  10  10  10  10  10
      10  10  10  10  10  10  10  10  10  10  10  10  10  10  10  10
      01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
      01  01  01  01  01  01  01  10  01  01  01  01  01  01  01  02
      02  02  02  02  02  02  02  02  02  02  02  02  02  02  02  02
      02  02  02  02  02  02  02  10  02  02  02  02  02  02  02  02
    
    # to_lower array (must have 256 elements)
      00  01  02  03  04  05  06  07  08  09  0A  0B  0C  0D  0E  0F
      10  11  12  13  14  15  16  17  18  19  1A  1B  1C  1D  1E  1F
      20  21  22  23  24  25  26  27  28  29  2A  2B  2C  2D  2E  2F
      30  31  32  33  34  35  36  37  38  39  3A  3B  3C  3D  3E  3F
      40  61  62  63  64  65  66  67  68  69  6A  6B  6C  6D  6E  6F
      70  71  72  73  74  75  76  77  78  79  7A  5B  5C  5D  5E  5F
      60  61  62  63  64  65  66  67  68  69  6A  6B  6C  6D  6E  6F
      70  71  72  73  74  75  76  77  78  79  7A  7B  7C  7D  7E  7F
      80  81  82  83  84  85  86  87  88  89  8A  8B  8C  8D  8E  8F
      90  91  92  93  94  95  96  97  98  99  9A  9B  9C  9D  9E  9F
      A0  A1  A2  A3  A4  A5  A6  A7  A8  A9  AA  AB  AC  AD  AE  AF
      B0  B1  B2  B3  B4  B5  B6  B7  B8  B9  BA  BB  BC  BD  BE  BF
      E0  E1  E2  E3  E4  E5  E6  E7  E8  E9  EA  EB  EC  ED  EE  EF
      F0  F1  F2  F3  F4  F5  F6  D7  F8  F9  FA  FB  FC  FD  FE  DF
      E0  E1  E2  E3  E4  E5  E6  E7  E8  E9  EA  EB  EC  ED  EE  EF
      F0  F1  F2  F3  F4  F5  F6  F7  F8  F9  FA  FB  FC  FD  FE  FF
    
    # to_upper array (must have 256 elements)
      00  01  02  03  04  05  06  07  08  09  0A  0B  0C  0D  0E  0F
      10  11  12  13  14  15  16  17  18  19  1A  1B  1C  1D  1E  1F
      20  21  22  23  24  25  26  27  28  29  2A  2B  2C  2D  2E  2F
      30  31  32  33  34  35  36  37  38  39  3A  3B  3C  3D  3E  3F
      40  41  42  43  44  45  46  47  48  49  4A  4B  4C  4D  4E  4F
      50  51  52  53  54  55  56  57  58  59  5A  5B  5C  5D  5E  5F
      60  41  42  43  44  45  46  47  48  49  4A  4B  4C  4D  4E  4F
      50  51  52  53  54  55  56  57  58  59  5A  7B  7C  7D  7E  7F
      80  81  82  83  84  85  86  87  88  89  8A  8B  8C  8D  8E  8F
      90  91  92  93  94  95  96  97  98  99  9A  9B  9C  9D  9E  9F
      A0  A1  A2  A3  A4  A5  A6  A7  A8  A9  AA  AB  AC  AD  AE  AF
      B0  B1  B2  B3  B4  B5  B6  B7  B8  B9  BA  BB  BC  BD  BE  BF
      C0  C1  C2  C3  C4  C5  C6  C7  C8  C9  CA  CB  CC  CD  CE  CF
      D0  D1  D2  D3  D4  D5  D6  D7  D8  D9  DA  DB  DC  DD  DE  DF
      C0  C1  C2  C3  C4  C5  C6  C7  C8  C9  CA  CB  CC  CD  CE  CF
      D0  D1  D2  D3  D4  D5  D6  F7  D8  D9  DA  DB  DC  DD  DE  FF
    
    # sort_order array (must have 256 elements)
      00  01  02  03  04  05  06  07  08  09  0A  0B  0C  0D  0E  0F
      10  11  12  13  14  15  16  17  18  19  1A  1B  1C  1D  1E  1F
      20  21  22  23  24  25  26  27  28  29  2A  2B  2C  2D  2E  2F
      30  31  32  33  34  35  36  37  38  39  3A  3B  3C  3D  3E  3F
      40  41  42  43  44  45  46  47  48  49  4A  4B  4C  4D  4E  4F
      50  51  52  53  54  55  56  57  58  59  5A  5B  5C  5D  5E  5F
      60  41  42  43  44  45  46  47  48  49  4A  4B  4C  4D  4E  4F
      50  51  52  53  54  55  56  57  58  59  5A  7B  7C  7D  7E  7F
      80  81  82  83  84  85  86  87  88  89  8A  8B  8C  8D  8E  8F
      90  91  92  93  94  95  96  97  98  99  9A  9B  9C  9D  9E  9F
      A0  A1  A2  A3  A4  A5  A6  A7  A8  A9  AA  AB  AC  AD  AE  AF
      B0  B1  B2  B3  B4  B5  B6  B7  B8  B9  BA  BB  BC  BD  BE  BF
      41  41  41  41  5C  5B  5C  43  45  45  45  45  49  49  49  49
      44  4E  4F  4F  4F  4F  5D  D7  D8  55  55  55  59  59  DE  DF
      41  41  41  41  5C  5B  5C  43  45  45  45  45  49  49  49  49
      44  4E  4F  4F  4F  4F  5D  F7  D8  55  55  55  59  59  DE  FF

    The ctype array contains bit values, with one element for one character. The to_lower and to_upper arrays simply hold the uppercase and lowercase characters that correspond to each member of the character set. For example, to_lower['A'] contains a, while to_upper['z'] contains Z.

    The sort_order array indicates the order that characters are to be sorted (it usually corresponds to to_upper, in which case the sorting will be case insensitive. All of the arrays are indexed by character value, except ctype, which is indexed by character value + 1 (an old legacy).

  3. Add the new character set (martian.conf) to the CHARSETS_AVAILABLE and COMPILED_CHARSETS lists in the file configure.in.

  4. Reconfigure and recompile MySQL, and test the new character set.

If you’re brave enough to tackle adding a new complex character set, there are a few more steps to this process. See the MySQL documentation for what is required (as well as the documentation in the existing complex character sets: czech, gbk, sjis, and tis160).

Summary

To understand how to get the most out of your database server, it’s important to understand the number of options you have when fine-tuning the server. To see how an existing server has been set up, use the SHOW VARIABLES statement, as well as SHOW STATUS to see how it’s been handling. The output of these two statements can reveal many hidden problems, including queries that are not optimized, poor use of available memory, or simply that it’s time for an upgrade.

MySQL supplies four configuration files that can help to get better performance from the server. Just choose the closest of my-huge.cnf, my-large.cnf, my-medium.cnf or my-small.cnf for your server situation.

Two of the easiest and most important variables to tweak are table_cache (the number of tables MySQL can keep open) and the key_buffer_size (how much of the indexes MySQL can keep in memory, minimizing disk access).

InnoDB databases have their own vagaries and work in a fundamentally different way than MyISAM tables, where each table is related to specific files. InnoDB configuration requires careful planning because disk space is allocated in advance.

Hardware too can be an easy way of improving the performance of a server, with memory, CPU, and disks being of primary importance.

MySQL comes with a benchmark suite, which can be used to compare the performance of various platforms, including other databases.

MySQL was developed in Scandinavia and has had good support for other languages besides English. It is easy to display error messages in other languages or add a character set.

Tagged with:
Aug 04

A regular part of a database administrator’s job is to do preventative maintenance, as well as to repair things when they go wrong. In spite of the best efforts, data errors can occur, such as in the case of a power failure that interrupts a write. Usually you can correct these fairly painlessly.

There are four main tasks involved in checking and repairing:

  • Optimizing tables

  • Analyzing tables (analyzes and stores the key distribution for MyISAM and BDB tables)

  • Checking tables (checks the tables for errors, and, for MyISAM tables, updates the key statistics)

  • Repairing tables (repairs corrupted MyISAM tables)

Optimizing Tables

Tables that contain BLOB and VARCHAR fields will, over time, become less optimized. Because these field types are variable in length, when records are updated, inserted, or deleted, they will not always take the same amount of space, the records will start to become fragmented, and empty spaces will remain. Just like with a fragmented disk, this situation will slow performance, so to keep MySQL in tiptop shape, you should regularly defragment it. The way to do this is to optimize the table, which can be done in a number of ways. There’s the OPTIMIZE TABLE statement, the mysqlcheck utility (if the server is running), or the myisamchk utility (if the server is not running or there is no interaction with the table).

Optimizing currently works only with MyISAM and partially with BDB tables. With MyISAM tables, optimizing does the following:

  • Defragments tables where rows are split or have been deleted

  • Sorts the indexes if they have not been already

  • Updates the index statistics if they have not been already

With BDB tables, optimizing analyzes the key distribution (the same as ANALYZE TABLE; see the "Analyzing Tables with ANALYZE TABLE" section later in this chapter).

Optimizing Tables with the OPTIMIZE Statement

The OPTIMIZE statement is a SQL statement used when connected to a MySQL database. The syntax is as follows:

OPTIMIZE TABLE tablename

You can also optimize many tables at once, separating each with a comma:

mysql> OPTIMIZE TABLE customer,sales;
+------------------+----------+----------+-----------------------------+
| Table            | Op       | Msg_type | Msg_text                    |
+------------------+----------+----------+-----------------------------+
| firstdb.customer | optimize | status   | Table is already up to date |
| firstdb.sales    | optimize | status   | OK                          |
+------------------+----------+----------+-----------------------------+
2 rows in set (0.02 sec)

The customer table in this instance has already been optimized.

Optimizing Tables with mysqlcheck

mysqlcheck is a command-line utility that can perform numerous checking and repairing tasks besides optimization. A full description of all the mysqlcheck features follows later in the chapter in the section titled "Using mysqlcheck." The server must be running for you to use mysqlcheck. To optimize the customer table from the firstdb database, use the -o mysqlcheck option, as follows:

% mysqlcheck -o firstdb customer  -uroot -pg00r002b
firstdb.customer           Table is already up to date

mysqlcheck allows you to optimize more than one table at a time by listing all the tables after the database name:

% mysqlcheck -o firstdb customer sales  -uroot -pg00r002b
firstdb.customer           Table is already up to date
firstdb.sales              Table is already up to date

You could also optimize the entire database by leaving out any table references, with the following:

% mysqlcheck -o firstdb -uroot -pg00r002b

Optimizing Tables with myisamchk

Finally, you can use the myisamchk command-line utility when the server is down or not interacting with the server. (Flush the tables before running this statement if the server is up with mysqladmin flush-tables. You still need to make sure the server is not interacting with the table, though, or else corruption may result.) This is the oldest way of checking tables. You have to run myisamchk from the exact location of the table or specify the path leading to the table. A full description of all the myisamchk features follows later in this chapter in the section titled "Using myisamchk."

The equivalent of an optimize statement is as follows:

myisamchk --quick --check-only-changed --sort-index --analyze tablename

or as follows:

myisamchk -q -C -S -a tablename

For example:

% myisamchk --quick --check-only-changed --sort-index --analyze customer
- check key delete-chain
- check record delete-chain
- Sorting index for MyISAM-table 'customer'

The -r option repairs the table, but also eliminates wasted space:

% myisamchk -r sales   
- recovering (with sort) MyISAM-table 'sales'
Data records: 8
- Fixing index 1
- Fixing index 2

If you do not specify the path to the table index file, and you’re not in the right directory, you’ll get the following error:

% myisamchk -r customer
myisamchk: error: File 'customer' doesn't exist

Specifying the full path to the .MYI file corrects this:

% myisamchk -r /usr/local/mysql/data/firstdb/customer
- recovering (with keycache) MyISAM-table '/usr/local/mysql/data/firstdb/customer'
Data records: 0

Warning

Tables are locked during the optimization, so don’t run this during peak hours! Also, make sure you have a reasonable amount of free space on the system when running OPTIMIZE TABLE. If you try to run it when your system has almost or already run out of disk space, MySQL may not be able to complete the optimization, leaving your table unusable.

Optimizing is an important part of any administrative routine for databases that contain MyISAM tables and should be performed regularly.

Analyzing Tables

Analyzing tables improves performance by updating the index information for a table so that MySQL can make a better decision on how to join tables. The distribution of the various index elements is stored for later usage. (Analyzing currently only works with MyISAM and BDB tables.)

There are three ways to analyze a table:

  • When connected to MySQL, with the ANALYZE TABLE statement

  • With the mysqlcheck command-line utility

  • With the myisamcheck command-line utility

Regular analysis of tables can help performance and should be a regular part of any maintenance routine.

Analyzing Tables with ANALYZE TABLE

ANALYZE TABLE is a statement used when connected to a database on the server. The syntax is as follows:

ANALYZE TABLE tablename

For example:

mysql> ANALYZE TABLE sales;
+---------------+---------+----------+----------+
| Table         | Op      | Msg_type | Msg_text |
+---------------+---------+----------+----------+
| firstdb.sales | analyze | status   | OK       |
+---------------+---------+----------+----------+
1 row in set (0.00 sec)

The Msg_type (message type) is one of status, error, info, or warning. Here’s what would happen if the index file was missing altogether and you tried to analyze the table:

mysql> ANALYZE TABLE zz;
+------------+---------+----------+----------------------------------+
| Table      | Op      | Msg_type | Msg_text                         |
+------------+---------+----------+----------------------------------+
| firstdb.zz | analyze | error    | Table 'firstdb.zz' doesn't exist |
+------------+---------+----------+----------------------------------+
1 row in set (0.00 sec)

The table will only be analyzed again if it has changed since the last time it was analyzed:

mysql> ANALYZE TABLE sales;
+---------------+---------+----------+-----------------------------+
| Table         | Op      | Msg_type | Msg_text                    |
+---------------+---------+----------+-----------------------------+
| firstdb.sales | analyze | status   | Table is already up to date |
+---------------+---------+----------+-----------------------------+
1 row in set (0.00 sec)
Analyzing Tables with mysqlcheck

The mysqlcheck command-line utility is discussed fully later in this chapter in the section titled "Using mysqlcheck." The server needs to be running for you to use mysqlcheck, and it works only with MyISAM tables. To use it to analyze tables, you use the -a option:

% mysqlcheck -a firstdb sales -uroot -pg00r002b
firstdb.sales                                      OK

You can also analyze more than one table from a database by listing the tables after the database name:

% mysqlcheck -a firstdb sales customer -uroot -pg00r002b
firstdb.sales                 Table is already up to date
firstdb.customer              Table is already up to date

If you tried to analyze a table that does not support analysis (such as an InnoDB table), no harm would be done and the operation would just fail. For example:

% mysqlcheck -a firstdb innotest -uroot -pg00r002b
firstdb.innotest
error    : The handler for the table doesn't support check/repair

You could also analyze all tables in the database with by leaving out any table names:

% mysqlcheck -a firstdb innotest -uroot -pg00r002b

Analyzing Tables with myisamchk

The myisamchk command-line utility is discussed fully later in this chapter in the section titled "Using myisamchk." The server should either not be running, or you must be sure that there is no interaction with the tables with which you’re working. If the –skip-external-locking option is not on, you can safely use myisamchk to analyze tables, even if the server is running. The tables will be locked, affecting access, but there will be no erroneous reports. If –skip-external-locking is used, you’ll need to flush the tables before starting the analysis (with mysqladmin flush-tables) and ensure that there is no access. You may get invalid results if mysqld or anything else accesses the table while myisamchk is running. To analyze tables, use the -a option:

% myisamchk -a /usr/local/mysql/data/firstdb/sales   
Checking MyISAM file: /usr/local/mysql/data/firstdb/sales
Data records:       9   Deleted blocks:       0
- check file-size
- check key delete-chain
- check record delete-chain
- check index reference
- check data record references index: 1
- check data record references index: 2

Checking Tables

Errors can occur when the indexes are not synchronized with the data. System crashes or power failures can all cause situations where the tables have become corrupted. Corruption of the data is fairly rare; in most cases, the corruption is of the index files. These can be hard to spot, though you may notice information being returned slowly or data not being found that should be there. Checking tables should be the first thing you do when you suspect an error. Some of the symptoms of corrupted tables include errors such as the following:

  • Unexpected end of file.

  • Record file is crashed.

  • tablename.frm is locked against change.

  • Can’t find file tablename.MYI (Errcode: ###).

  • Got error ### from table handler. The perror utility gives more information about the error number. Just run perror (which is stored in the same directory as the other binaries such as mysqladmin) and the error number. For example:

    % perror 126
    126 = Index file is crashed / Wrong file format

Some of the other more common errors include:

126 = Index file is crashed / Wrong file format
127 = Record-file is crashed
132 = Old database file
134 = Record was already deleted (or record file crashed)
135 = No more room in record file
136 = No more room in index file
141 = Duplicate unique key or constraint on write or update
144 = Table is crashed and last repair failed
145 = Table was marked as crashed and should be repaired

Once connected to the MySQL server, you can issue a CHECK TABLE command, make use of the mysqlcheck utility (when the server is running), or use the myisamchk utility when the server has been stopped. Checking updates the index statistics and checks for errors.

If any errors are found, the table will need to be repaired (see the "Repairing Tables" section later in this chapter). Serious errors mark the table as corrupt, in which case it can no longer be used until it is repaired.

Tip

Always check tables after a power failure or a system crash. You can usually fix any corruption that has occurred before users notice any problems.

Checking Tables with CHECK TABLES

The syntax for CHECK TABLE is as follows:

CHECK TABLE tablename [option]

For example:

mysql> CHECK TABLE customer;
+------------------+-------+----------+----------+
| Table            | Op    | Msg_type | Msg_text |
+------------------+-------+----------+----------+
| firstdb.customer | check | status   | OK       |
+------------------+-------+----------+----------+
1 row in set (0.01 sec)

CHECK can check only MyISAM and InnoDB tables.

There are five options depending on the level of checking you want to do, as shown in Table 4

Table 4: CHECK TABLE Options

Option

Description

QUICK

This is the quickest check and does not scan the rows to check for wrong links.

FAST

Only checks tables that haven’t been closed properly.

CHANGED

Only checks tables that haven’t been closed properly or have been changed since last check.

MEDIUM

The default option. It scans rows to check that deleted links are correct. It also calculates a key checksum for the rows and verifies this with a calculated checksum for the keys.

EXTENDED

This is the slowest method, but it checks the table for complete consistency by doing a full key lookup for every index associated with each row.

 

The QUICK option is useful for checking tables where you don’t suspect any errors.

If an error or warning is returned, you should try and repair the table.

You can check more than one table at a time by listing the tables one after another,

for example:

mysql> CHECK TABLE sales,customer;
+------------------+-------+----------+----------+
| Table            | Op    | Msg_type | Msg_text |
+------------------+-------+----------+----------+
| firstdb.sales    | check | status   | OK       |
| firstdb.customer | check | status   | OK       |
+------------------+-------+----------+----------+
2 rows in set (0.01 sec)

Checking Tables with mysqlcheck

The mysqlcheck command-line utility can be used when the server is running and works only with MyISAM tables. It is described fully later in this chapter in the section titled "Using mysqlcheck." Table 5 lists the options.

The syntax is as follows:

mysqlcheck [options] databasename tablename[s]

For example:

% mysqlcheck -c firstdb customer -uroot -pg00r002b
firstdb.customer                               OK

Table 5: The mysqlcheck Options That Apply to Table Checking

Option

Description

–auto-repair

Used in conjunction with one of the check options, it will automatically beginto repair corrupted tables after the checks have completed.

-c, –check

Checks tables.

-C, –check-only-changed

Checks tables that have changed since the last check or were not closed properly.

-F, –fast

Checks tables that haven’t been closed properly.

-e, –extended

This is the slowest form for checking, but it will make sure the table is completely consistent. You can also use this option to repair, though itisusually not necessary.

-m, –medium-check

This is much faster than the extended check, and it finds the vast majority oferrors.

-q, –quick

The fastest check, this does not check table rows when checking. When repairing, it only repairs the index tree.

 

You can check more than one table by listing a number of tables after the database name:

% mysqlcheck -c firstdb sales customer -uroot -pg00r002b
firstdb.sales                                      OK
firstdb.customer                                   OK

You can check all tables in the database by just specifying the name of the database.

% mysqlcheck -c firstdb -uroot -pg00r002b

Checking Tables with myisamchk

When the server is shut down or there is no interaction with the tables you’re checking, you can use the myisamchk command-line option (described fully in the "Using myisamchk" section later in this chapter). If the –skip-external-locking option is not on, you can safely use myisamchk to check tables, even if the server is running. The tables will be locked, affecting access, but there will be no erroneous reports. If –skip-external-locking is used, you’ll need to flush the tables before starting the check (with mysqladmin flush-tables) and ensure that there is no access. You may get wrong results (with tables being marked as corrupted even when they are not) if mysqld or anything else accesses the table while myisamchk is running.

The syntax is as follows:

mysiamchk [options] tablename

The equivalent to the CHECK TABLE statement is the medium option:

myisamchk -m table_name 

The default for myisamchk is the ordinary check option (-c). There is also the fast check (-F), which only checks tables that haven’t been closed properly. This is not the same as the lowercase -f option, which is the force option, meaning the check continues even if errors occur. There is also the medium check (-m), slightly slower and more complete. The most extreme option is the -e option (that performs an extended check), which is the most thorough and slowest option. It’s also usually a sign of desperation; use this only when all other options have failed. Increasing the key_buffer_size variable can speed up the extended check (if you have enough memory). See Table 6 for the checking options.

Table 6: myisamchk Checking Options

Option

Description

-c, –check

Ordinary check and the default option.

-e, –extend-check

Slowest and most thorough form of check. If you are using –extended-check and have much memory, you should increase the value of key_buffer_size a lot!

-F, –fast

Fast check, which only checks tables that haven’t been closed properly.

-C, –check-only-changed

Checks only the tables that have been changed since the last check.

-f, –force

This runs the repair option if any errors are found in the table.

-i, –information

Displays statistics about the table that is checked.

-m, –medium-check

Medium check, faster than an extended check, and good enough for most cases.

-U, –update-state

Keeps information about when the table was checked and whether the table has crashed, which is useful for the -C option. Should not be used when the table is being used and the –skip-external-locking option is active.

-T, –read-only

Does not mark the table as checked (useful for running myisamchk when the server is active and the –skip-external-locking option is in use).

 

The following is a sample myisamchk output when errors are found:

% myisamchk largetable.MYI
Checking MyISAM file: Hits.MYI
Data records: 2960032   Deleted blocks:       0
myisamchk: warning: 1 clients is using or hasn't closed the table
properly
- check file-size
myisamchk: warning: Size of datafile is: 469968400 Should be: 469909252
- check key delete-chain
- check record delete-chain
- check index reference
- check data record references index: 1
- check data record references index: 2
- check data record references index: 3
myisamchk: error: Found 2959989 keys of 2960032
- check record links
myisamchk: error: Record-count is not ok; is 2960394 Should be: 2960032
myisamchk: warning: Found    2960394 parts   Should be: 2960032 parts

Repairing Tables

If you have checked the tables and errors have been found, you’ll need to repair them. There are various repair options available, depending on which method you use, but you may not have success. If the disk has failed, or if none of them work, the only option is to restore from your backup. Repairing a table can take up significant resources, both disk and memory:

  • Generally, repairing a table takes up twice as much disk space as the original data file (on the same disk). A quick repair (see the options in the following sections) is an exception because the data file is not modified.

  • Some space for the new index file (on the same disk as the original). The old index is deleted at the start, so this is usually not significant, but it will be if the disk is close to full.

  • With the standard and –sort-recover options, a sort buffer is created. This takes up the following amount of space (largest_key + row_pointer_length) * number_of_rows * 2. You can move some or all of this to memory (and increase the speed of the process) by increasing the size of the mysqld variable sort_buffer_size if you have the available memory. Otherwise, it is created as specified by the TMPDIR environment variable or the -t myisamchk option.

  • Memory usage is determined by the mysqld variables or the options set in the myisamchk command line (see the section titled "Using myisamchk").

If the error is caused by the table running out of space and the table type is InnoDB, you will have to enlarge the InnoDB tablespace. MyISAM tables have a huge theoretical size limit (eight million terabytes), but by default pointers are only allocated for 4GB. If the table reaches this limit, you can extend it by using the MAX_ROWS and AVG_ROW_LENGTH ALTER TABLE parameters. To prepare the table called limited for great things (currently it only has three records), you use the following:

mysql> ALTER TABLE limited MAX_ROWS=999999999999 AVG_ROW_LENGTH=100;
Query OK, 3 rows affected (0.28 sec)
Records: 3  Duplicates: 0  Warnings: 0

This allocates pointers for a much greater number of records. The AVG_ROW_LENGTH is used when BLOB and TEXT fields are present, and it gives MySQL an idea of the average size of a record, which it can then use for optimization purposes.

Repairing Non-MyISAM Table Types

The three methods of repairing discussed in the following sections work only with MyISAM tables. Some of the options have been reported to work occasionally with BDB tables, but they were not designed for this. Currently, the only way to repair corrupted BDB and InnoDB tables is to restore from backup.

Repairing Tables with REPAIR TABLE

You can run the REPAIR TABLE statement when connected to the MySQL server. It currently only works with MyISAM tables. See Table 7 for the options.

The syntax is as follows:

Table 10: Available REPAIR TABLE Options

Option

Description

QUICK

Fastest repair because the data file is not modified. It uses much less disk space as well because the data file is not modified.

EXTENDED

Attempts to recover every possible row from the data file. This option should not be used unless as a last resort because it may produce garbage rows.

USE_FRM

This is the option to use if the .MYI file is missing or has a corrupted header. It will rebuild the indexes from the definitions found in the .frm table definition file.

 

The following is an example of using REPAIR in the case of a missing .MYI file. Let’s delete the .MYI file of an existing table, t4.

% ls -l t4.*
-rw-rw----  1 mysql  mysql    10 Jun 14 02:00 t4.MYD
-rw-rw----  1 mysql  mysql  4096 Jun 14 02:00 t4.MYI
-rw-rw----  1 mysql  mysql  8550 Jun  8 10:46 t4.frm
% rm t4.MYI
% mysql -uguru2b -pg00r002b firstdb

A normal REPAIR does not work:

mysql> REPAIR TABLE t4;
+------------+--------+----------+--------------------------------------+
| Table      | Op     | Msg_type | Msg_text                             |
+------------+--------+----------+--------------------------------------+
| firstdb.t4 | repair | error    | Can't find file: 't4.MYD' (errno: 2) |
+------------+--------+----------+--------------------------------------+
1 row in set (0.47 sec)

The current error message (4.0.3) indicates that the .MYD file cannot be found, when it’s actually the .MYI file that’s missing. The error message is likely to have been clarified by the time you read this. To repair the table in this instance, you need to use the USE_FRM option, which, as the name suggests, uses the .frm definition file to re-create the .MYI index file:

mysql> REPAIR TABLE t4 USE_FRM;
+------------+--------+----------+------------------------------------+
| Table      | Op     | Msg_type | Msg_text                           |
+------------+--------+----------+------------------------------------+
| firstdb.t4 | repair | warning  | Number of rows changed from 0 to 2 |
| firstdb.t4 | repair | status   | OK                                 |
+------------+--------+----------+------------------------------------+
2 rows in set (0.46 sec)

Repairing Tables with mysqlcheck

The mysqlcheck command-line utility is used while the server is still running and works only with MyISAM tables. It is described fully later in this chapter in the "Using mysqlcheck" section. To repair tables, you use the -r option:

% mysqlcheck -r firstdb customer -uroot -pg00r002b
firstdb.customer                                   OK

You can repair multiple tables at the same time by listing the table names after the

database name:

% mysqlcheck -r firstdb customer sales -uroot -pg00r002b
firstdb.customer                                   OK
firstdb.sales                                      OK

If for some reason all tables in a database are corrupt, you can repair them all by just supplying the database name:

% mysqlcheck -r firstdb -uroot -pg00r002b
Repairing Tables with myisamchk

You can use the myisamchk command-line utility (described fully later in this chapter in the "Using myisamchk" section) repair tables (see Table 10.8).

Table 8: Repairing Tables with myisamchk

Option

Description

-D #, –data-file-length=#

Specifies the maximum length of the data file when re-creating it.

-e, –extend-check

Attempts to recover every possible row from the data file. This option should not be used unless as a last resort because it may produce garbage rows.

-f, –force

Overwrites old temporary files (that have an extension of .TMD) instead of aborting if it encounters a preexisting one.

-k #, keys-used=#

Specifies which keys to use, which can make the process faster. Each binary bit stands for one key starting at 0 for the first key.

-r, –recover

Repairs most corruption and should be the first option attempted. You can increase thesort_buffer_size to make the recovery go more quickly if you have the memory. This option will not recover from the rare form of corruption where a unique key is not unique.

-o, –safe-recover

A more thorough, yet slower repair option than -r that should be used only if -r fails. This reads through all rows and rebuilds the indexes based on the rows. It also uses less disk space than -r because a sort buffer is not created. You can increase the size of key_ buffer_size to improve repair speed.

-n, –sort-recover

Forces MySQL to use sorting to resolve the indexes, even if the resulting temporary files are large.

–character-sets-dir=…

The directory containing the character sets.

–set-character-set=name

Specifies a new character set for the index.

-t, –tmpdir=path

Specifies a new path for storing temporary files if you don’t want to use whatever the TMPDIR environment variable specifies.

-q, –quick

Fastest repair because the data file is not modified. Specifying the q twice (-q -q) will modify the data file if there are duplicate keys. Uses much less disk space as well because the data file is not modified.

-u, –unpack

Unpacks a file that has been packed with the myisampack utility.

 

The server should either not be running, or you must be sure there is no interaction with the tables with which you’re working, such as when you start MySQL with the –skip-external-locking option. If the –skip-external-locking option is not on, you can only safely use myisamchk to repair tables if you are sure there will be no simultaneous access. Whether –skip-external-locking is used or not, you’ll need to flush the tables before starting the repair (with mysqladmin flush-tables) and ensure that there is no access. You may get wrong results (with tables being marked as corrupted even when they are not) if mysqld or anything else accesses the table while myisamchk is running.

The syntax is as follows:

myisamchk [options] [tablenames]

You must run myisamchk from the directory containing the .MYI files or supply the path. The following examples show a repair in action, with MySQL deciding whether to use sorting or a keycache:

% myisamchk -r customer
- recovering (with keycache) MyISAM-table 'customer.MYI'
Data records: 0
% myisamchk -r sales   
- recovering (with sort) MyISAM-table 'sales.MYI'
Data records: 9
- Fixing index 1
- Fixing index 2

If you have lots of memory, besides increasing the size of sort_buffer_size and key_buffer_size as described previously, you can also set some other variables to make myisamchk perform more snappily. See the full myisamchk description later in this chapter in the "Using myisamchk" section.

Using mysqlcheck

The mysqlcheck utility is a boon to more recent users of MySQL because, beforehand, much of the repairing and checking functionality could only be used when the server was shut down. Luckily this limitation is a thing of the past with the mysqlcheck utility.

mysqlcheck uses the CHECK, REPAIR, ANALYZE, and OPTIMIZE statements to perform these tasks from the command line, which is useful for automated maintenance of your databases (see Table 9).

The syntax is as follows:

mysqlcheck [options] databasename [tablenames]

or as follows:

mysqlcheck [options] --databases databasename1 [databasename2 databasename 3 ...]

or as follows:


Table 9: Mysqlcheck Options

Option

Description

-A, –all-databases

Checks all available databases.

-1, –all-in-1

Combines queries for tables into one query per database (instead of one per table). Tables are in a comma-separated list.

-a, –analyze

Analyzes the listed tables.

–auto-repair

Automatically repairs tables if they are corrupted (after all tables in the query have been checked).

-#, –debug=…

Outputs a debug log.

–character-sets-dir=…

This specifies the directory where the character sets are.

-c, –check

Checks tables.

-C, –check-only-changed

Checks tables that have changed since the last check or were not closed properly.

–compress

Uses compression in the client/server protocol.

-?, –help

Displays the help message and exits.

-B, –databases

Lists a number of databases to check (all tables in the databases are checked).

–default-character-set=…

Sets the default character set.

-F, –fast

Checks tables that haven’t been closed properly.

-f, –force

Forces the process to continue even if it encounters an error.

-e, –extended

This is the slowest form for checking but will make sure the table is completely consistent. You can also use this option to repair, though it is usually not necessary.

-h, –host=…

Hostname to which to connect.

-m, –medium-check

Much faster than the extended check and finds the vast majority of errors.

-o, –optimize

Optimizes the tables.

-p, –password[=...]

The password with which to connect.

-P, –port=…

The port to use for connecting.

-q, –quick

The fastest check, this does not check table rows when checking. When repairing, it only repairs the index tree.

-r, –repair

Repairs most errors, except unique keys that somehow contain duplicates.

-s, –silent

Displays no output except for error messages.

-S, –socket=…

Specifies the socket file to use when connecting.

–tables

List of tables to check. With the -B option, this will take precedence.

-u, –user=#

Specifies the user to connect as.

-v, –verbose

Prints lots of output about the process.

-V, –version

Displays the version information and exits.

The mysqlcheck utility also has a feature that allows it to be run in different ways without specifying the options. By simply creating a copy of mysqlcheck with one of the following names, it will take that default behavior:

  • mysqlrepair: The default option is -r.

  • mysqlanalyze: The default option is -a.

  • mysqloptimize: The default option is -o.

The default option when it is named mysqlcheck is -c. All of these renamed files can have the full mysqlcheck functionality—it’s just that their default behavior is changed.

Using myisamchk

The myisamchk utility is the older utility, available since the early days of MySQL. It is also used to analyze, check, and repair tables, but care needs to be taken if you want to use it when the server is running. Table 10 describes the general myisamchk options, Table 11 describes the check options, Table 12 describes the repair options, and Table 13 describes other options.

The server should either not be running, or you must be sure there is no interaction with the tables with which you’re working, such as when you start MySQL with the –skip-external-locking option. If the –skip-external-locking option is not on, you can only safely use myisamchk to repair tables if you are sure there will be no simultaneous access. Whether or not –skip-external-locking is used, you’ll need to flush the tables before starting the repair (with mysqladmin flush-tables) and ensure that there is no access.

I suggest you rather use one of the other options if the server is running.

The syntax is as follows:

myisamchk [options] tablename[s]

You must run myisamchk from the directory where the .MYI index files are located unless you specify the path to them; otherwise you’ll get the following error:

% myisamchk -r sales.MYI
myisamchk: error: File 'sales.MYI' doesn't exist

Specifying the path solves the problem:

% myisamchk -r /usr/local/mysql/data/firstdb/sales.MYI
- recovering (with sort) MyISAM-table '/usr/local/mysql/data/firstdb/sales.MYI'
Data records: 9
- Fixing index 1
- Fixing index 2

The table name can be specified with or without the .MYI extension.

% myisamchk -r sales
- recovering (with sort) MyISAM-table 'sales'
Data records: 9
- Fixing index 1
- Fixing index 2
% myisamchk -r sales.MYI
- recovering (with sort) MyISAM-table 'sales.MYI'
Data records: 9
- Fixing index 1
- Fixing index 2

Warning

A common mistake is to try run myisamchk on an .MYD data file. Always use the .MYI index file!

You can use wildcard character to search all tables in a database directory (*.MYI) or even all tables in all databases:

Table 10: General myisamchk Options

Option

Description

-#, –debug=debug_options

Outputs a debug log. A common debug_option string is d:t:o,filename.

-?, –help

Displays a help message and exits.

-O var=option, –set-variable var=option

Sets the value of a variable. The possible variables and their default values for myisamchk can be examined with myisamchk –help.

-s, –silent

Only outputs error messages. A second s can be used to completely silence myisamchk.

-v, –verbose

Displays more information than usual. As with silent, multiple v’s can be used to output more information (-vv or -vvv).

-V, –version

Displays the myisamchk version details and exits.

-w, –wait

If the table is locked, -w will wait for the table to be unlocked rather than exiting with an error. If mysqld was running with the –skip-external-locking option, the table can only be locked by another myisamchk command.

 

By running myisamchk –help, besides the general options, you can see what variables you can change with the -O option and what the current settings are:

% myisamchk --help
..
Possible variables for option --set-variable (-O) are:
key_buffer_size       current value: 520192
myisam_block_size     current value: 1024
read_buffer_size      current value: 262136
write_buffer_size     current value: 262136
sort_buffer_size      current value: 2097144
sort_key_blocks       current value: 16
decode_bits           current value: 9
ft_min_word_len       current value: 4
ft_max_word_len       current value: 254
ft_max_word_len_for_sort  current value: 20

The space allocated by the key_buffer_size is used when doing an extended check or when indexes are inserted one row at a time (using the safe-recover option). sort_buffer_size is used in the default repair, when indexes are sorted in the repair.

To achieve a faster repair, set the sort_buffer_size to about one-quarter of the total available memory. Only one of the two variables is used at a time, so you don’t need to worry about running out of memory by making both values large.

Note

Inside the my.cnf (or my.ini) file, there are separate sections for mysqld and myisamchk. You can quite easily set the sort_buffer_size to a high value for repairing, and keep it lower if your system has other requirements for day-to-day running.


Table 11: myisamchk Check Options

Option

Description

-c, –check

Ordinary check and the default option.

-e, –extend-check

Slowest and most thorough form of check. If you are using –extended-check and don’t have much memory, you should increase the value of key_buffer_size a lot!

-F, –fast

Fast check that only checks tables that haven’t been closed properly.

-C, –check-only-changed

Checks only the tables that have been changed since the last check.

-f, –force

This runs the repair option if any errors are found in the table.

-i, –information

Displays statistics about the table that is checked.

-m, –medium-check

Medium check, faster than an extended check and good enough for most cases.

-U, –update-state

Keeps information about when the table was checked and whether the table has crashed, which is useful for the -C option. Should not be used when the table is being used and the –skip-external-locking option is active.

-T, –read-only

Does not mark the table as checked (useful for running myisamchk when the server is active and the –skip-external-locking option is in use).

Table 12: myisamchk Repair Options

Option

Description

-D #, –data-file-length=#

Specifies the maximum length of the data file when re-creating it.

-e, –extend-check

Attempts to recover every possible row from the data file. This option should not be used unless as a last resort because it may produce garbage rows.

-f, –force

Overwrites old temporary files (that have an extension of .TMD) instead of aborting if it encounters a preexisting one.

-k #, keys-used=#

Specifies the keys to use, which can make the process faster. Each binary bit stands for one key starting at 0 for the first key (for example, 1 is the first index, 10 is the second index).

-r, –recover

Repairs most corruption and should be the first option attempted. You can increase the sort_buffer_size to make the recover go more quickly if you have the memory. This option will not recover from the rare form of corruption where a unique key is not unique.

-o, –safe-recover

A more thorough, slower repair option than -r, which should be used only if -r fails. This reads through all rows and rebuilds the indexes based on the rows. It also uses less disk space than -r because a sort buffer is not created. You can increase the size of key_ buffer_size to improve repair speed.

-n, –sort-recover

Forces MySQL to uses sorting to resolve the indexes, even if the resulting temporary files are large.

–character-sets-dir=…

The directory containing the character sets.

–set-character-set=name

Specifies a new character set for the index.

-t, –tmpdir=path

Specifies a new path for storing temporary files if you don’t want to use the contents of the TMPDIR environment variable.

-q, –quick

Fastest repair as the data file is not modified. Running this option a second time will modify the data file if there are duplicate keys. Uses much less disk space as well because the data file is not modified.

-u, –unpack

Unpacks a file that has been packed with the myisampack utility.

Table 13: Other myisamchk Options

Option

Description

-a, –analyze

Analyzing tables improves performance by updating the index information for atable so that MySQL can make a better decision on how to join tables. The distribution of the various index elements is stored for later usage. This option isthe same as ANALYZE TABLE.

-d, –description

Displays a description of the table.

-A, –set-auto-increment[=value]

Sets the AUTO_INCREMENT counter to the specified value (or increments it by one if no value is supplied).

-S, –sort-index

Sorts the index tree blocks in descending order, which improves the performance of seeks and table scanning by key.

-R, –sort-records=#

Sorts records according to the index specified (index numbers begin from 1; you can use SHOW INDEX to see an ordered list). This can speed up queries that are ordered on this index, as well as ranged selects. It will probably be very slow if you sort a large table for the first time.

Running myisamchk with the -d option produces the following kind of output:

% myisamchk -d customer
MyISAM file:         customer
Record format:       Packed
Character set:       latin1 (8)
Data records:        3  Deleted blocks:                 0
Recordlength:        75
table description:
Key Start Len Index   Type
1      2         4      unique   long
Tagged with:
preload preload preload