GSP
Quick Navigator

Search Site

Unix VPS
A - Starter
B - Basic
C - Preferred
D - Commercial
MPS - Dedicated
Previous VPSs
* Sign Up! *

Support
Contact Us
Online Help
Handbooks
Domain Status
Man Pages

FAQ
Virtual Servers
Pricing
Billing
Technical

Network
Facilities
Connectivity
Topology Map

Miscellaneous
Server Agreement
Year 2038
Credits
 

USA Flag

 

 

Man Pages


Manual Reference Pages  -  GALERA (8)

NAME

galera - Galera Documentation \$1 \n[an-margin] level \n[rst2man-indent-level] level margin: \n[rst2man-indent\n[rst2man-indent-level]] - \n[rst2man-indent0] \n[rst2man-indent1] \n[rst2man-indent2]

 Galera Cluster for MySQL is a synchronous multi-master database cluster, based on Oracle\(aqs MySQL/InnoDB. The MySQL forks from Percona and MariaDB also provide Galera clustered versions as  Percona XtraDB Cluster and  MariaDB Galera Cluster respectively. [1] Galera is 100% GPL open source technology and is integrated with MySQL Server by Codership Oy.

To read more about Galera, you can continue reading about_galera_cluster. If you just want to quickly install and start your first cluster, you should jump to howto/start_your_first_galera_cluster.

This is the official documentation for Galera Cluster for MySQL and covers both Codership\(aqs Galera Cluster for MySQL, as well as Percona XtraDB Cluster and MariaDB Galera Cluster. Where Percona or MariaDB differ or contain additional features and functionality, this is specifically highlighted througout the manual.

This manual was first released in conjunction with Galera 3.0 in 2013, however we have made an attempt to cover feature compatibility back to Galera 1.0.

In other words, you should be able to use this reference manual for all Galera versions and together with all major MySQL variants (forks).

CONTENTS

Other (upstream) Mysql Manuals
Contents
     About Galera Cluster for MySQL
     Galera Overview
     Galera benefits
     About Codership Oy
Footnotes
     Quickstart and other HowTos
     HowTo: Start your first Galera Cluster
     What you need
     Installing
     Starting a Galera cluster
     Testing that the cluster works
     HowTo: Deploy a Galera Cluster into EC2 using Severalnines ClusterControl
     Step 1. EC2 Security Groups
     Step 2. Start the Nodes
     Step 3. Set Up the Keypair
     Step 4. Generate Deployment Package
     Step 5. Deploy from Control Instance
     Installing
     Installing Galera Cluster for MySQL from RPM packages
     Installing Galera Cluster for MySQL from DEB packages
     Installing Galera Cluster for MySQL from TAR binary packages
     Installing Percona XtraDB Cluster from RPM packages
     Installing Percona XtraDB Cluster from DEB packages
     Installing Percona XtraDB Cluster from TAR binary packages
     Installing MariaDB Galera Cluster from RPM packages
     Installing MariaDB Galera Cluster from DEB packages
     Installing MariaDB Galera Cluster from TAR binary packages
     Other 3rd party installation packages
     Compiling from source
     Configuration and Monitoring
     Configuring Linux for Galera
     Recommended MySQL configuration
     MySQL WSREP configuration options
     Galera configuration options
     MySQL WSREP status variables
     Configuring Galera for Vagrant
     Startup and Administration
     Starting the cluster
     State Snapshot Transfer (SST)
     Incremental State Transfer (IST)
     Backups
     Rolling Upgrades
     Rolling Schema Upgrade
     Understanding Galera
     Galera Cluster for MySQL internals
     Different Galera Architectures
     Load balancers
     General notes about using Galera Cluster with a load balancer
     Mixing Galera with MySQL replication
     Migrating to Galera
     Incompatibilities with standard (single node) MySQL
     Frequently Asked Questions
     Legal and Meta information
     External Resources
     Release notes
     Galera Cluster for MySQL releases
     Galera Replication releases
Indices And Tables
Footnotes
Author
Copyright

OTHER (UPSTREAM) MYSQL MANUALS

This manual only covers the Galera replication library and the "Write Set Replication (wsrep) patches" added to mysqld itself. Standard MySQL features are only covered if relevant to using Galera, such as how to use certain configuration options or how to mix Galera clustering and classic MySQL replication.

Other standard MySQL functionality, if it is not related to Galera, is not covered by this manual, rather the reader should refer to the following official documentations for each product:
o  MySQL Reference Manual
o  Percona Server Documentation
o  MariaDB AskMonty Knowledgebase

CONTENTS

    About Galera Cluster for MySQL

Galera Cluster for MySQL is a synchronous, multi-master cluster for the MySQL database. You can read and write to any node, and you can lose any one node without interruption in operations, and without the need to handle complicated failover procedures. [image]

    Galera Overview

Galera Replication is a general purpose replication library for any transactional system. It can be used to create a synchronous multi-master replication solution, such as to achieve high availability and scale-out.

The first and most popular Galera based solution is Galera Cluster for MySQL. On a high level, this is standard MySQL, and standard InnoDB storage engine, using the Galera replication library as a replication plugin. To be precise, we do extend the MySQL replication plugin API to provide all the information and hooks needed for truly multi-master, synchronous replication. We call this API the Write Set REPlication API. [1] To learn more about the internals of Galera, WSREP API and how they interact with MySQL, you should read understanding/internals.

To use Galera Cluster for MySQL, you need to download a version of MySQL that includes the WSREP patches. Galera enabled versions are available for all major MySQL forks: Oracle MySQL, Percona Server and MariaDB. The chapter installing contains links to download each of these.

Galera implements so called certification based replication. This concept was introduced in, and Galera was inspired by, the academic research of Fernando Pedone. The basic idea is that the transaction to be replicated - the write set - not only contains the database rows to replicate, it also includes information about all the locks that were held by the database (ie. InnoDB) during the transaction. Each node then certifies the replicated write set against other write sets in the applier queue, and if there are no conflicting locks, we know that the write set can be applied. At this point the transaction is considered committed, after which each node continues to apply it to the InnoDB tablespace.

This approach is also called virtually synchronous replication since it is logically synchronous, but actual writing (and committing) to the InnoDB tablespace happens independently (and thus, strictly speaking, asynchronously) on each node.

    Galera benefits

Galera is a significant step in terms of improving MySQL high availability. The MySQL ecosystem has offered users various ways to achieve high availability, but typically all alternatives have provided only some of the following features, but not all, thus making the choice of a high availability solution an exercise in tradeoffs.

Galera provides all of the following features:
o True multi-master. Read and write to any node at any time.
o Synchronous replication. No data is lost at node crash or failover.
o Tightly coupled. All nodes hold the same state. No diverged data between nodes allowed.
o Multi-threaded slave, for any workload. No slave lag or performance overhead.
o No master-slave failover operations or use of VIP necessary.
o Hot standby. No downtime during failover (since there is no failover).
o Automatic node provisioning. No need to manually back up the database and copy it to the new node.
o Supports InnoDB storage engine. Compatible with MySQL installed base.

The result is a high availability solution that is both robust in terms of data integrity and high performance with instant failovers. The following diagram illustrates how different common database clustering solutions compare in these dimensions: [image]

The other main benefit of Galera is good support for typical cloud usage. Automatic node provisioning makes elastic scale-out and scale-in operations painless. Galera has proven to perform very well in the cloud, such as when using multiple small node instances, across multiple data centers (e.g. AWS zones) or even over Wider Area Network.

    About Codership Oy

Galera was created by Codership Oy, a company based in Helsinki, Finland.

Codership was founded and development on Galera was started in 2007 by the three founders Seppo Jaakola, Teemu Ollakka and Alex Yurchenko. Development was funded by doing consulting work, and later by startup grants and loans from Finnish government agency Tekes. Before starting work on Galera, the founders had already worked on 3 previous clustering solutions for MySQL, the last of which, Tungsten, is also in widespread use today.

Galera 1.0 was eventually released in October 2011. At this point there were already several customers running Galera Cluster for MySQL in production on the beta versions 0.7 and 0.8.

The word "Codership" is based on the word "coder" / "coders". Think of it as similar to the word "friendship".

FOOTNOTES

[1] To use the Galera replication library in some other database or transactional application than MySQL, you would similarly implement this WSREP API.

    Quickstart and other HowTos

This section contains HowTo documents. They are simple recipes to get going with your Galera Cluster without explaining too much on each step.

    HowTo: Start your first Galera Cluster

    What you need

You need 3 hosts running a recent Linux distribution.

NOTE: With the instructions given here, various Galera features will default to using the first network interface. For this reason, the below instructions are not sufficient for running Galera under Vagrant Virtualbox instances. See /configuration_and_monitoring/configuring_galera_for_vagrant.

In the below, we will use the hostnames: host1, host2, host3.

Make sure there is no firewall between the nodes and that selinux or apparmor are disabled or running in permissive mode.

    Installing

This example uses Ubuntu Precise and apt-get. For other Linux distributions steps are similar, for details see: /installing.

Easiest way to install Galera is with a package manager like apt or yum. As of this writing Codership doesn\(aqt provide apt and yum repositories, so to keep things simple we will install Percona XtraDB Cluster  from Percona\(aqs repositories.

Do the following on all hosts:
:: sudo su gpg --keyserver hkp://keys.gnupg.net --recv-keys 1C4CBDCDCD2EFD2A gpg -a --export CD2EFD2A | apt-key add - echo deb http://repo.percona.com/apt precise main > /etc/apt/sources.list.d/percona.list echo deb-src  http://repo.percona.com/apt precise main >> /etc/apt/sources.list.d/percona.list apt-get update apt-get install -y percona-xtradb-cluster-server-5.5 percona-xtradb-cluster-client-5.5 percona-xtradb-cluster-galera-2.x /etc/init.d/mysql start

NOTE: Change the 2 occurences of "precise" with the version of your own Ubuntu or Debian installation. (e.g. "quantal" or "squeeze"...)

NOTE: When asked for a MySQL root password, in this example we have pressed enter to leave it empty.

You can check that the installation was successful with a command like this:


mysql -e "SHOW VARIABLES LIKE \(aqversion_%\(aq"


Which will output:


+-------------------------+------------------------------------------------+
| Variable_name           | Value                                          |
+-------------------------+------------------------------------------------+
| version_comment         | Percona XtraDB Cluster (GPL), wsrep_23.7.r3821 |
| version_compile_machine | i686                                           |
| version_compile_os      | Linux                                          |
+-------------------------+------------------------------------------------+


    Starting a Galera cluster

You have now successfully installed and started Percona XtraDB Cluster on your 3 hosts. But the MySQL servers are not yet connected to each other as a cluster, and the Galera library is not even loaded in the configuration. The MySQL servers are simply running as 3 independent databases.

We will use the default State Snapshot Transfer method, which is mysqldump. For this it is necessary to create a mysql client connection between the nodes. We need to grant root privileges in the MySQL user database to allow that:

On all nodes do the following: mysql GRANT ALL ON . TO  \(aqroot\(aq@\(aqhost1\(aq; GRANT ALL ON . TO  \(aqroot\(aq@\(aqhost2\(aq; GRANT ALL ON . TO  \(aqroot\(aq@\(aqhost3\(aq; exit

Starting the first node is special. Do the following on host1 only:


cp /usr/share/mysql/wsrep.cnf /etc/mysql/my.cnf


Edit /etc/mysql/my.cnf with your favorite editor (mine is nano):


nano /etc/mysql/my.cnf


Find the following options and set these values for them:


wsrep_provider="/usr/lib/libgalera_smm.so"
wsrep_cluster_address="gcomm://"


The "gcomm:://" cluster address tells MySQL that it is the first node in the cluster. It will start operating immediately without trying to connect to any other nodes. You should only use this when starting the first cluster node for the first time!

Save the changes, and start MySQL:


/etc/init.d/mysql restart


Check that startup succeeded and changes are applied:


mysql -e "SHOW VARIABLES LIKE \(aqwsrep_cluster_address\(aq"

+-----------------------+----------+ | Variable_name | Value | +-----------------------+----------+ | wsrep_cluster_address | gcomm:// | +-----------------------+----------+

Immediately after startup, open my.cnf in your editor again, and change the value of wsrep_cluster_address so that it points to the other two nodes:


wsrep_cluster_address="host2,host3"


NOTE: Do not restart MySQL at this point.

WARNING: You should never leave wsrep_cluster_address="gcomm://" in your my.cnf configuration file. Always change it so that it lists the other nodes in your cluster. For more information, see option_wsrep_cluster_address.

Do the following on host2:


cp /usr/share/mysql/wsrep.cnf /etc/mysql/my.cnf


Edit /etc/mysql/my.cnf. Find the following options and set these values for them:


wsrep_provider="/usr/lib/libgalera_smm.so"
*wsrep_cluster_address="host1,host3"*


Restart MySQL: /etc/init.d/mysql restart

Do the following on host3:


cp /usr/share/mysql/wsrep.cnf /etc/mysql/my.cnf


Edit /etc/mysql/my.cnf. Find the following options and set these values for them:


wsrep_provider="/usr/lib/libgalera_smm.so"
*wsrep_cluster_address="host1,host2"*


Restart MySQL: /etc/init.d/mysql restart

    Testing that the cluster works

You should now have a Galera cluster up and running!

On any node, connect to MySQL:


mysql


You can verify that all nodes have connected by checking the following status variables:


show status like \(aqwsrep_%\(aq;

+----------------------------+--------------------------------------+ | Variable_name | Value | +----------------------------+--------------------------------------+ ... | wsrep_local_state_comment | Synced (6) | | wsrep_cluster_size | 3 | | wsrep_ready | ON | +----------------------------+--------------------------------------+

In the above we see that this node is Synced ie it is connected to the cluster and operational. There are a total of 3 nodes in the cluster. The wsrep_ready status variable also tells us that this node is connected to the cluster and able to handle transactions.

We can now create a test table and insert some data.

On host1 open a MySQL prompt and do:


CREATE DATABASE galeratest;
use galeratest
CREATE TABLE t (id INT PRIMARY KEY auto_increment, msg TEXT);
INSERT INTO t (msg) VALUES ("Hello my dear cluster");
INSERT INTO t (msg) VALUES ("Hello again");


On host2, see if the data was replicated correctly:


use galeratest
SELECT * FROM t;

+----+-----------------------+ | id | msg | +----+-----------------------+ | 3 | Hello my dear cluster | | 6 | Hello again | +----+-----------------------+

It worked!

(Notice that the auto_increment id\(aqs are not 1 and 2. This is a Galera feature, it is ok. See option_wsrep_auto_increment_control)

    HowTo: Deploy a Galera Cluster into EC2 using Severalnines ClusterControl

Severalnines produces ClusterControl, a tool to easily deploy, monitor and manage MySQL clusters. With Severalnines ClusterControl you can deploy a full Galera Cluster with a single command. ClusterControl will make SSH connections to each node, install and start the necessary software.

    Step 1. EC2 Security Groups

Galera node needs to be accessible at several ports, so for simplicity it is better to allow all incoming ICMP and TCP packets from other nodes in the same security group. The default EC2 security group allows exactly this. You should also allow inbound http and ssh access, at least to the first node, which we will call Control Instance below.

    Step 2. Start the Nodes

Start 4 or more of your favourite EC2 instances (this howto was prepared with RightImage_CentOS_5.6_x64_v5.7.14). You will need to choose one of the instances to run a control server and that may be anything, the rest of the instances will be Galera nodes and for best results they should be identical.

    Step 3. Set Up the Keypair

Copy the keypair used to start the instances to the control instance, it will be used by deployment script to install and configure the rest of the nodes:


scp -i keypair keypair root@<control_instance_public_dns_name>:


    Step 4. Generate Deployment Package

Go to  http://www.severalnines.com/galera-configurator/

Page 1 “General Settings”: specify cloud provider (Amazon EC2), number of instances, mysql root password, OS user (root)

Page 2 “Storage”: specify EC2 instance RAM size, planned db size, number of cores

Page 3 “Individual Nodes”:
o For ClusterControl server specify local IP address and the absolute path to keypair set up in step 3 (in this case /root/keypair).
o For Galera servers specify local IP addresses and the desirable location of data directory (RightScale images mounts local storage under /mnt)

After entering the required information you will be sent a tarball containing deployment scripts and there also will be a personal HTTP link at Severalnines site.

    Step 5. Deploy from Control Instance

This is how deployment goes. (Note that you\(aqll get a different tarball link)


[root@<control instance> ~]# wget http://www.severalnines.com/galera-configurator/tmp/mv00aikh65qmoju9p1qqlfaoo6/s9s-galera-1.0.0-rpm.tar.gz
[root@<control instance> ~]# tar -xzf s9s-galera-1.0.0-rpm.tar.gz
[root@<control instance> ~]# cd ./s9s-galera-1.0.0-rpm/mysql/scripts/install
[root@<control instance> install]# ./deploy.sh


Answer a couple of questions and sit back while ClusterControl connects to each EC2 instance, installs Galera and starts the cluster.

In a few minutes your cluster will be up and running. Success will be signified by an invitation to visit  http://<control_instance_public_dns_name>/cmon/ to check how the cluster is doing.

NOTE: You might want to set up HTTP password to protect the monitor from unauthorized access.

    Installing

TODO : Overview of available options

    Installing Galera Cluster for MySQL from RPM packages

TODO

    Installing Galera Cluster for MySQL from DEB packages

TODO

    Installing Galera Cluster for MySQL from TAR binary packages

TODO

    Installing Percona XtraDB Cluster from RPM packages

TODO

    Installing Percona XtraDB Cluster from DEB packages

TODO

    Installing Percona XtraDB Cluster from TAR binary packages

TODO

    Installing MariaDB Galera Cluster from RPM packages

TODO

    Installing MariaDB Galera Cluster from DEB packages

TODO

    Installing MariaDB Galera Cluster from TAR binary packages

TODO

    Other 3rd party installation packages

TODO

    Compiling from source

TODO

    Configuration and Monitoring

TODO: intro/summary

    Configuring Linux for Galera

TODO

    Recommended MySQL configuration

TODO

    MySQL WSREP configuration options

TODO

    wsrep_auto_increment_control

wsrep_auto_increment_control doc here...

    wsrep_cluster_address

wsrep_cluster_address documentation here...

    Galera configuration options

TODO

    MySQL WSREP status variables

TODO

    3rd party tools

TODO

    Configuring Galera for Vagrant

TODO

    Startup and Administration

This section explains in detail how to start a Galera Cluster, including the available State Snapshot Transfer options. Then we cover various administrative tasks such as taking a backup, rolling upgrades and rolling schema upgrades (DDL).

    Starting the cluster

TODO

    State Snapshot Transfer (SST)

TODO

    Incremental State Transfer (IST)

TODO

    Backups

TODO

    Rolling Upgrades

TODO

    Rolling Schema Upgrade

TODO

    Understanding Galera

TODO

Sub sections in this chapter are still completely missing, but topics to cover will be: pc, quorum minimal cluster size  http://www.codership.com/wiki/doku.php?id=info total order / gtid commit sequence

    Galera Cluster for MySQL internals

TODO

Use the picture from Henrik\(aqs "introduction to Galera" here.

Discuss wsrep api and link to launchpad project

    Different Galera Architectures

In this chapter we present some commonly used system architectures for Galera Cluster. From the understanding_galera chapter you should already be familiar with the concepts of primary component, quorum and resulting need for at minimum 3 nodes. (However, we also consider the special case when you only have 2 nodes available, and your options at dealing with that.)

Most commonly Galera is used with some load balancer. Often the load balancer functionality can even be built into your MySQL client library - this is currently the case with the official JDBC and PHP MySQL clients. However, we also cover some architectures where a load balancer is not used.

Galera has been found to perform very well also when used over Wider Area Networks, even across contintents. Using Galera over WAN is also covered here.

.seealso:: load_balancers

TODO: subsections (toctree)

    Load balancers

    General notes about using Galera Cluster with a load balancer

TODO: Basically: Catch exceptions, then retry

TODO: toctree: jdbc client, php client, hw load balancer, haproxy, glb, libglb.so

    Mixing Galera with MySQL replication

TODO

    Migrating to Galera

TODO: incorporate henrik\(aqs blog post on the topic. Explain with pictures online migration over mysql replication.

    Incompatibilities with standard (single node) MySQL

TODO

    Frequently Asked Questions

TODO

    Legal and Meta information

TODO (toctree)
 
o license information: Both Galera and the documentation
o trademark attributions
o about Sphinx RST documentation format / building the docs from source

    External Resources

Congratulations, you have now reached the end of our fine manual. This chapter is a list of further Galera resources you may find useful:

TODO copy from wiki + others community mailing lists, forums, stackoverflow, etc...

    Release notes

TODO (Release notes have been sent to the mailing list:  https://groups.google.com/forum/#!forum/codership-team)

    Galera Cluster for MySQL releases

    Galera Replication releases

INDICES AND TABLES

o genindex
o modindex
o search

FOOTNOTES

[1] "MySQL" is a registered trademark of Oracle Corporation. "Percona XtraDB Cluster" and "Percona Server" are registered trademarks of Percona LLC. "MariaDB" and "MariaDB Galera Cluster" are registered trademarks of Monty Program Ab. See legal_and_meta_information for more information.

AUTHOR

Codership Oy

COPYRIGHT

2012, Codership Oy
Search for    or go to Top of page |  Section 8 |  Main Index


23.2.1 GALERA (8) April 03, 2016

Powered by GSP Visit the GSP FreeBSD Man Page Interface.
Output converted with manServer 1.07.