You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
兔子 6536853c69 bug fix 3 months ago
test init 1 year ago
.gitignore init 1 year ago
LICENSE init 1 year ago init 1 year ago
biglong_trx.txt init 1 year ago
binlog.go init 1 year ago
binlog_status.txt init 1 year ago
binlog_test.go init 1 year ago
dump_binlog_from_pos.go init 1 year ago
dump_binlog_from_pos_test.go init 1 year ago
event.go init 1 year ago
get_all_gtid_of_binlog_dir.go init 1 year ago
get_all_gtid_of_binlog_dir_test.go init 1 year ago
get_first_previous_gtid_of_binlog_dir.go init 1 year ago
get_first_previous_gtid_of_binlog_dir_test.go init 1 year ago
get_gtid_of_binlog.go init 1 year ago
get_gtid_of_binlog_test.go init 1 year ago
get_previous_gtid.go init 1 year ago
get_previous_gtid_test.go init 1 year ago
get_unexecuted_binlog_files_by_gtid.go init 1 year ago
get_unexecuted_binlog_files_by_gtid_test.go init 1 year ago
get_unexecuted_binlog_pos_by_gtid.go init 1 year ago
get_unexecuted_binlog_pos_by_gtid_test.go init 1 year ago
go.mod update 3 months ago
go.sum update 3 months ago
logger.go init 1 year ago
parse.go bug fix 3 months ago
parse_test.go update 1 year ago
util.go init 1 year ago
writer.go init 1 year ago


Some utilities for mysql binlog

WARNING: these utilities are tested for mysql 5.5.33, but are not tested for mysql 5.6+

Dump binlog from pos

DumpBinlogFromPos(srcFilePath string, startPos int, targetFilePath string)

This function will dump binlog (at srcFilePath) from pos (startPos), and the output is at targetFilePath

If the startPos is 0 or 4, the whole binlog will be dump. Otherwise, the source binlog header (including FORMAT_DESCRIPTION_EVENT & ROTATE_EVENT & PREVIOUS_GTIDS_LOG_EVENT) will be dump as the target binlog header, and then the source data will be dump from startPos

This will make target binlog complete and available to replay.

Rotate relay log

RotateRelayLog(relayLogPath string, endPos int)

This function will add a rotate event to a relay log (at relayLogPath), after the position (endPos), and truncate the data after the position (endPos)

WARNING: after manually rotate relay log, DONOT forget to update relay-log.index

Fake master server

NewFakeMasterServer(port int, unusedServerId int, characterSet int, keepAliveWhenFinish bool, baseDir string)

WARNING: only support mysql 5.5.x

This fake master server is very helpful if you want to replay some binlog files to a mysql instance, and you're afraid of mysqlbinlog (, for example)

Mysql replication is more reliable way to replay binlog, what we need is :

  1. server := NewFakeMasterServer(...)
  2. server.Start()
  3. In target mysql instance, change master to the fake server, and start slave
  4. the server will be closed when done or error
  5. you can abort the server by server.Abort()

####Some other features:

  1. start slave until is also supported
  2. large packet (>= 1<<24-1 bytes) is supported
  3. multiple binlog files are supported, fake server will act as a real replication master (rotate to the next when one is finished)


Argument _
port the fake server port
unusedServerId the fake server id, should not be duplicate with any other mysql instance
characterSet the fake server character set id, should be the same with target mysql instance's. You can get the id by SELECT id, collation_name FROM information_schema.collations ORDER BY id
keepAliveWhenFinish when false, the fake server will quit when all binlogs are replayed. when true, the fake server will wait for more binlogs.
baseDir where the binlog files are located

Get unexecuted binlog files by gtid

GetUnexecutedBinlogFilesByGtid(binlogDir string, binlogBaseName string, executedGtidDesc string) (ret []string, err error)

This function will search binlog files in binlogDir, for the one contains a event whose gtid is not contained in executedGtidDesc

The scenario is in mysql replication, if mysql master is broken, mysql slave can call GetUnexecutedBinlogFilesByGtid to search in mysql master binlog dir, for the binlog files which need to be replay in slave

Get unexecuted binlog pos by gtid

GetUnexecutedBinlogPosByGtid(binlogFilePath string, executedGtidDesc string) (pos uint, err error)

This function will search first binlog event pos, which is not contained in executedGtidDesc, in binlog file (binlogFilePath)

Or return EOF error if has no unexecuted binlog event

####Pull requests and issues are warmly welcome