codema.in
Tue 17 Jan 2017 5:26PM

mysql replication is not working (backup not uptodate)

PP Pirate Praveen Public Seen by 363

We need to reinitialize the replication I think. The backup server was down, I restarted it, but replication is showing an error.

show slave status gave this error.

Last_IO_Errno: 1236
                Last_IO_Error: Got fatal error 1236 from master when reading data from binary log: 'Could not find first log file name in binary log index file'
               Last_SQL_Errno: 1594
               Last_SQL_Error: Relay log read failure: Could not parse relay log event entry. The possible reasons are: the master's binary log is corrupted (you can check this by running 'mysqlbinlog' on the binary log), the slave's relay log is corrupted (you can check this by running 'mysqlbinlog' on the relay log), a network problem, or a bug in the master's or slave's MySQL code. If you want to check the master's binary log or slave's relay log, you will be able to know their names by issuing 'SHOW SLAVE STATUS' on this slave.
  Replicate_Ignore_Server_Ids: 
             Master_Server_Id: 3

Any volunteers to reset the replication? We'll have to copy the master db using rsync and restart the replication from new copy.

PP

Pirate Praveen Tue 17 Jan 2017 5:32PM

Some more info about the error https://www.percona.com/blog/2014/10/08/mysql-replication-got-fatal-error-1236-causes-and-cures/ basically we need to recreate replication as required log files got removed from master (older logs get cleared automatically).

DU

[deactivated account] Fri 20 Jan 2017 5:10PM

I will help in setting up the backup.

PP

Pirate Praveen Sat 21 Jan 2017 1:40PM

I have shared access with you. See http://fosscommunity.in/index.php?title=Poddery#Setting_up_Backup for steps to setup backup and also getting encrypted access file.

DU

[deactivated account] Sun 22 Jan 2017 2:44PM

Completed backup. Slave is now up and running.

PP

Pirate Praveen Sun 22 Jan 2017 2:53PM

Thanks for the help! Happy to see more hands helping. I have also added a ping test to uptime robot account so we will get notified if backup server goes down again. We will have to 10 days to fix things before the master server will clear the binary logs.

If you get a notification that Poddery Backup is down, just login to scaleway.com and see if the server is okay and "cold boot" if the server is hanged. We need to also find the root cause, @mintojoseph can you help?

MJ

Minto Joseph Sun 22 Jan 2017 5:17PM

Will check this tomorrow.