cluster failover

   2-3. Goldilocks Cluster 주요 기능

        1) replica

        2) shard

        3) failover

        4) online scale out

        5) (etc) global index, sequence

📕Goldilocks 교육 커리큘럼 - LINUX, CLUSTER

---

3) Cluster failover

Cluster group 내의 모든 멤버는 같은 데이터를 갖지만, Shared Nothing Disk 구조이기 때문에 노드 장애 시 다른 노드를 통해 지속적인 운영이 가능하다.  

즉 1by2로 구성된 클러스터에서 G1N1 장비에 장애가 발생하더라도, 동일한 데이터를 갖는 G1N2로 failover가 진행된다.

Failover는 Cluster의 Failover Thread를 통해 진행되며, 특정 노드나 네트워크 장애 시 Member에 대한 Offline 및 Coordinator 재선정 등의 Failover가 처리된다.

• G1N1 Shutdown 시

[2019-01-29 13:33:12.569597 INSTANCE(GOLDILOCKS) THREAD(25991,139630611420928)] [WARNING]

[CDISPATCHER-0] finalize receiver socket(member pos=0), last errno=11, by discon command(0)

(GOLDILOCKS) ERR-HY000(58010): broken socket

[2019-01-29 13:33:12.569592 INSTANCE(GOLDILOCKS) THREAD(25991,139630517151488)] [WARNING]

[CDISPATCHER-1] finalize receiver socket(member pos=0), last errno=11, by discon command(0)

(GOLDILOCKS) ERR-HY000(58010): broken socket

[2019-01-29 13:33:12.669908 INSTANCE(GOLDILOCKS) THREAD(25991,139630611420928)] [WARNING]

[CDISPATCHER] set connection(target member pos=0); physical connection(0), consensus(1)

(GOLDILOCKS) ERR-HY000(58010): broken socket

[2019-01-29 13:33:12.669944 INSTANCE(GOLDILOCKS) THREAD(25991,139630611420928)] [INFORMATION]

[CDISPATCHER-0] finalize sender socket(member pos=0)

[2019-01-29 13:33:12.669935 INSTANCE(GOLDILOCKS) THREAD(25940,140005945087744)] [INFORMATION]

[FAILOVER] begin - failover member(0)

[2019-01-29 13:33:12.670525 INSTANCE(GOLDILOCKS) THREAD(25940,140005945087744)] [INFORMATION]

[FAILOVER] acquire failover lock

[2019-01-29 13:33:12.670605 INSTANCE(GOLDILOCKS) THREAD(25940,140005945087744)] [INFORMATION]

[FAILOVER] domain coordinator failover - all members in group (1) are inactive.

[2019-01-29 13:33:12.670617 INSTANCE(GOLDILOCKS) THREAD(25940,140005945087744)] [INFORMATION]

[FAILOVER] global scn of member(1) is ‘2,0,7'

[2019-01-29 13:33:12.670623 INSTANCE(GOLDILOCKS) THREAD(25940,140005945087744)] [INFORMATION]

[FAILOVER] global coordinator failover - new global coordinator member(1)

[2019-01-29 13:33:12.670661 INSTANCE(GOLDILOCKS) THREAD(25940,140005945087744)] [INFORMATION]

disconnct member - local member pos(1), forward member pos(1), target member pos(0)

[2019-01-29 13:33:12.706747 INSTANCE(GOLDILOCKS) THREAD(25991,139630498268928)] [INFORMATION]

[CDISPATCHER-0] disconnecting member - driver member pos (1), target member pos (0)

[2019-01-29 13:33:12.806889 INSTANCE(GOLDILOCKS) THREAD(25991,139630498268928)] [WARNING]

[CDISPATCHER] set connection(target member pos=0); physical connection(0), logical connection(0), consensus(0)

[2019-01-29 13:33:12.806904 INSTANCE(GOLDILOCKS) THREAD(25991,139630498268928)] [WARNING]

[CDISPATCHER] set connection(target member pos=0); physical connection(0), logical connection(0), consensus(0)

[2019-01-29 13:33:12.806948 INSTANCE(GOLDILOCKS) THREAD(25991,139630498268928)] [INFORMATION]

[CDISPATCHER-1] finalize sender socket(member pos=0)

[2019-01-29 13:33:12.907094 INSTANCE(GOLDILOCKS) THREAD(25940,140005945087744)] [INFORMATION]

[FAILOVER] member(0) has failovered

[2019-01-29 13:33:12.907136 INSTANCE(GOLDILOCKS) THREAD(25940,140005945087744)] [INFORMATION]

[COORDINATOR] failover domain coordinator - all members in group (1) are inactive

[2019-01-29 13:33:12.907155 INSTANCE(GOLDILOCKS) THREAD(25940,140005945087744)] [WARNING]

reset remote session map - member(0)

[2019-01-29 13:33:12.907296 INSTANCE(GOLDILOCKS) THREAD(25940,140005945087744)] [INFORMATION]

[FAILOVER] finished

[2019-01-29 13:33:12.993033 INSTANCE(GOLDILOCKS) THREAD(25940,140006861182720)] [INFORMATION]

[FAILOVER] offline member - member(0)

[2019-01-29 13:33:12.993087 INSTANCE(GOLDILOCKS) THREAD(25940,140006861182720)] [INFORMATION]

offline member - member(0)

[2019-01-29 13:33:12.993623 INSTANCE(GOLDILOCKS) THREAD(25940,140006861182720)] [INFORMATION]

end offline member

[2019-01-29 13:33:12.994138 INSTANCE(GOLDILOCKS) THREAD(25940,140006861182720)] [INFORMATION]

[FAILOVER] end offline member

gSQL> select * from x$cluster_connection;

ERR-HY000(16354): connection of member 'G1N1' is broken

---

Networks 단절

CLUSTER_HEARTBEAT_INTERVAL

CLUSTER_HEARTBEAT_RETRY_COUNT