MySQL Connect 2013 has been a great edition. There was of course a lot of nice announcements of improvements in the the core MySQL server technology. One of the major announcement that received a lot of buzz was MySQL Fabric. MySQL Fabric is an infrastructure component aimed at simplifying construction of a highly available, sharded, MySQL server based architecture.
Horizontal scale out for MySQL is a hard problem. The MySQL sharding requirement has up till now only be addressed in a non general solution way. Using MySQL at Scale remains a big challenge.
All the big actors of the web have been faced with this scale out issue. They all have developed their own tools/framework to address this need. The new cloud providers have also been faced with this requirement when trying to offer database as a service solutions around MySQL with transparent elasticity.
We can mention the following Sharding solutions davelopped by the key players of the web :
- Google/Youtube has built Vitess (in Go) and open sourced it
- Twitter has built Gizzard (in Scala) and open sourced it
- Tumblr has built Jetpants (in Ruby) and open sourced it
- Facebook has also it own sharding framework which is a range based model. The HA is addressed with MHA. they have presented their architecture at various events
Theses frameworks cover the needs:
- Master promotions,
- Cloning slaves,
- Supports a range-based sharding scheme for MySQL
- Rebalancing shards,
- Split a range-based shard into N new shards
High Availability is also addressed by most of these frameworks. MySQL Fabric also handles HA. All the improvements made around replication have helped handle correctly the HA part of the architecture in a more resilient way.
So now with the introduction of MySQL Fabric there is new sharding framework available. So what ? Interesting ?
Yes it is very interesting.
First, MySQL fabric has been written in python which in my opinion is a good point as devops love python.
Second, MySQL fabric has been released under the GPL license which is also a very good point.
Last MySQL Fabric is aimed as a general purpose sharding framework.
MySQL fabric is quite simple :
- A Fabric Server holding and serving the sharding metadata.
- A set of commands to do various action (split shard ,…)
- Specific connectors : Java, Python to request shard location to the Fabric server through XMLRPC call.
Two questions for me:
- What is the status of the PHP and C drivers regarding MySQL Fabric ?
- Is it possible to fully abstract the client code from the fact that the data is sharded. Some connector config could specify the fabric server. The client code would then not be changed at all.
A Brief Introduction to MySQL Fabric 2013-09-21 Mats Kindahl
MySQL Fabric – Sharding – Introduction 2013-09-21 VN (Narayanan Venkateswaran)
MySQL Fabric – Sharding – Simple Example 2013-09-22 VN (Narayanan Venkateswaran)
MySQL Fabric – Sharding – Shard Maintenance 2013-09-27 VN (Narayanan Venkateswaran)
MySQL Fabric – Sharding – Migrating From an Unsharded to a Sharded Setup 2013-09-22 VN (Narayanan Venkateswaran)
Installing MySQL Fabric on Windows 2013-10-03 Todd Farmer
MySQL 5.7 Fabric: any good? 2013-09-23 Ulf Wendel
Writing a Fault-tolerant Database Application using MySQL Fabric 2013-09-21 Alfranio Junior
MySQL Fabric support in Connector/Python 2013-09-22 Geert Vanderkelen
MySQL Connector/J with Fabric Support 2013-09-21 Jess Balint