ReplicationSource (Apache HBase 1.7.1 API)

java.lang.Object
- java.lang.Thread
- - org.apache.hadoop.hbase.replication.regionserver.ReplicationSource

All Implemented Interfaces:

Runnable, ReplicationSourceInterface
```
@InterfaceAudience.Private
public class ReplicationSource
extends Thread
implements ReplicationSourceInterface
```
Class that handles the source of a replication stream. Currently does not handle more than 1 slave For each slave cluster it selects a random number of peers using a replication ratio. For example, if replication ration = 0.1 and slave cluster has 100 region servers, 10 will be selected.
A stream is considered down when we cannot contact a region server on the peer cluster for more than 55 seconds by default.

Nested Class Summary

Nested Classes
Modifier and Type	Class and Description
`static class`	`ReplicationSource.LogsComparator` Comparator used to compare logs together based on their start time
`class`	`ReplicationSource.ReplicationSourceShipperThread`
`static class`	`ReplicationSource.WorkerState`

Nested classes/interfaces inherited from class java.lang.Thread
Thread.State, Thread.UncaughtExceptionHandler

Field Summary

Fields
Modifier and Type Field and Description

protected ReplicationSourceLogQueue logQueue
- Fields inherited from class java.lang.Thread
  MAX_PRIORITY, MIN_PRIORITY, NORM_PRIORITY

Fields
Modifier and Type	Field and Description
`protected ReplicationSourceLogQueue`	`logQueue`

Constructor Summary

Constructors
Constructor and Description

ReplicationSource()

Constructors
Constructor and Description
`ReplicationSource()`

Method Summary

Methods
Modifier and Type	Method and Description
`void`	`addHFileRefs(TableName tableName, byte[] family, List<Pair<org.apache.hadoop.fs.Path,org.apache.hadoop.fs.Path>> pairs)` Add hfile names to the queue to be replicated.
`void`	`enqueueLog(org.apache.hadoop.fs.Path log)` Add a log to the list of logs to replicate
`org.apache.hadoop.fs.Path`	`getCurrentPath()` Get the current log that's replicated
`org.apache.hadoop.fs.Path`	`getLastLoggedPath()`
`long`	`getLastLoggedPosition()`
`String`	`getPeerClusterId()` Get the id that the source is replicating to.
`String`	`getPeerClusterZnode()` Get the id that the source is replicating to
`Map<String,PriorityBlockingQueue<org.apache.hadoop.fs.Path>>`	`getQueues()`
`MetricsSource`	`getSourceMetrics()` Get the associated metrics.
`String`	`getStats()` Get a string representation of the current statistics for this source
`Map<String,ReplicationStatus>`	`getWalGroupStatus()` get the stat of replication for each wal group.
`void`	`init(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FileSystem fs, ReplicationSourceManager manager, ReplicationQueues replicationQueues, ReplicationPeers replicationPeers, Stoppable stopper, String peerClusterZnode, UUID clusterId, ReplicationEndpoint replicationEndpoint, MetricsSource metrics)` Instantiation method used by region servers
`protected boolean`	`isPeerEnabled()` check whether the peer is enabled or not
`boolean`	`isSourceActive()`
`void`	`run()`
`protected boolean`	`sleepForRetries(String msg, int sleepMultiplier)` Do the sleeping logic
`void`	`startup()` Start the replication
`void`	`terminate(String reason)` End the replication
`void`	`terminate(String reason, Exception cause)` End the replication
`void`	`terminate(String reason, Exception cause, boolean join)`

Methods inherited from class java.lang.Thread
activeCount, checkAccess, clone, countStackFrames, currentThread, destroy, dumpStack, enumerate, getAllStackTraces, getContextClassLoader, getDefaultUncaughtExceptionHandler, getId, getName, getPriority, getStackTrace, getState, getThreadGroup, getUncaughtExceptionHandler, holdsLock, interrupt, interrupted, isAlive, isDaemon, isInterrupted, join, join, join, resume, setContextClassLoader, setDaemon, setDefaultUncaughtExceptionHandler, setName, setPriority, setUncaughtExceptionHandler, sleep, sleep, start, stop, stop, suspend, toString, yield

Methods inherited from class java.lang.Object
equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

- Field Detail
  - logQueue
```
protected ReplicationSourceLogQueue logQueue
```
- Constructor Detail
  - ReplicationSource
```
public ReplicationSource()
```
- Method Detail
  - init
```
public void init(org.apache.hadoop.conf.Configuration conf,
        org.apache.hadoop.fs.FileSystem fs,
        ReplicationSourceManager manager,
        ReplicationQueues replicationQueues,
        ReplicationPeers replicationPeers,
        Stoppable stopper,
        String peerClusterZnode,
        UUID clusterId,
        ReplicationEndpoint replicationEndpoint,
        MetricsSource metrics)
          throws IOException
```
    Instantiation method used by region servers
    
    Specified by:
    
    init in interface ReplicationSourceInterface
    
    Parameters:
    conf - configuration to use
    fs - file system to use
    manager - replication manager to ping to
    stopper - the atomic boolean to use to stop the regionserver
    peerClusterZnode - the name of our znode
    clusterId - unique UUID for the cluster
    replicationEndpoint - the replication endpoint implementation
    metrics - metrics for replication source
    
    Throws:
    
    IOException - IO Exception
  - enqueueLog
```
public void enqueueLog(org.apache.hadoop.fs.Path log)
```
    Description copied from interface: ReplicationSourceInterface
    
    Add a log to the list of logs to replicate
    
    Specified by:
    
    enqueueLog in interface ReplicationSourceInterface
    
    Parameters:
    log - path to the log to replicate
  - getQueues
```
@InterfaceAudience.Private
public Map<String,PriorityBlockingQueue<org.apache.hadoop.fs.Path>> getQueues()
```
  - addHFileRefs
```
public void addHFileRefs(TableName tableName,
                byte[] family,
                List<Pair<org.apache.hadoop.fs.Path,org.apache.hadoop.fs.Path>> pairs)
                  throws ReplicationException
```
    Description copied from interface: ReplicationSourceInterface
    
    Add hfile names to the queue to be replicated.
    
    Specified by:
    
    addHFileRefs in interface ReplicationSourceInterface
    
    Parameters:
    tableName - Name of the table these files belongs to
    family - Name of the family these files belong to
    pairs - list of pairs of { HFile location in staging dir, HFile path in region dir which will be added in the queue for replication}
    
    Throws:
    
    ReplicationException - If failed to add hfile references
  - run
```
public void run()
```
    Specified by:
    
    run in interface Runnable
    
    Overrides:
    
    run in class Thread
  - sleepForRetries
```
protected boolean sleepForRetries(String msg,
                      int sleepMultiplier)
```
    Do the sleeping logic
    
    Parameters:
    msg - Why we sleep
    sleepMultiplier - by how many times the default sleeping time is augmented
    
    Returns:
    True if sleepMultiplier is < maxRetriesMultiplier
  - isPeerEnabled
```
protected boolean isPeerEnabled()
```
    check whether the peer is enabled or not
    
    Returns:
    true if the peer is enabled, otherwise false
  - startup
```
public void startup()
```
    Description copied from interface: ReplicationSourceInterface
    
    Start the replication
    
    Specified by:
    
    startup in interface ReplicationSourceInterface
  - terminate
```
public void terminate(String reason)
```
    Description copied from interface: ReplicationSourceInterface
    
    End the replication
    
    Specified by:
    
    terminate in interface ReplicationSourceInterface
    
    Parameters:
    reason - why it's terminating
  - terminate
```
public void terminate(String reason,
             Exception cause)
```
    Description copied from interface: ReplicationSourceInterface
    
    End the replication
    
    Specified by:
    
    terminate in interface ReplicationSourceInterface
    
    Parameters:
    reason - why it's terminating
    cause - the error that's causing it
  - terminate
```
public void terminate(String reason,
             Exception cause,
             boolean join)
```
  - getPeerClusterZnode
```
public String getPeerClusterZnode()
```
    Description copied from interface: ReplicationSourceInterface
    
    Get the id that the source is replicating to
    
    Specified by:
    
    getPeerClusterZnode in interface ReplicationSourceInterface
    
    Returns:
    peer cluster id
  - getPeerClusterId
```
public String getPeerClusterId()
```
    Description copied from interface: ReplicationSourceInterface
    
    Get the id that the source is replicating to.
    
    Specified by:
    
    getPeerClusterId in interface ReplicationSourceInterface
    
    Returns:
    peer cluster id
  - getCurrentPath
```
public org.apache.hadoop.fs.Path getCurrentPath()
```
    Description copied from interface: ReplicationSourceInterface
    
    Get the current log that's replicated
    
    Specified by:
    
    getCurrentPath in interface ReplicationSourceInterface
    
    Returns:
    the current log
  - getLastLoggedPath
```
public org.apache.hadoop.fs.Path getLastLoggedPath()
```
  - getLastLoggedPosition
```
public long getLastLoggedPosition()
```
  - isSourceActive
```
public boolean isSourceActive()
```
  - getStats
```
public String getStats()
```
    Description copied from interface: ReplicationSourceInterface
    
    Get a string representation of the current statistics for this source
    
    Specified by:
    
    getStats in interface ReplicationSourceInterface
    
    Returns:
    printable stats
  - getSourceMetrics
```
public MetricsSource getSourceMetrics()
```
    Description copied from interface: ReplicationSourceInterface
    
    Get the associated metrics.
    
    Specified by:
    
    getSourceMetrics in interface ReplicationSourceInterface
    
    Returns:
    The metrics for this source.
  - getWalGroupStatus
```
public Map<String,ReplicationStatus> getWalGroupStatus()
```
    Description copied from interface: ReplicationSourceInterface
    
    get the stat of replication for each wal group.
    
    Specified by:
    
    getWalGroupStatus in interface ReplicationSourceInterface
    
    Returns:
    stat of replication

Class ReplicationSource

Nested Class Summary

Nested classes/interfaces inherited from class java.lang.Thread

Field Summary

Fields inherited from class java.lang.Thread

Constructor Summary

Method Summary

Methods inherited from class java.lang.Thread

Methods inherited from class java.lang.Object

Field Detail

logQueue

Constructor Detail

ReplicationSource

Method Detail

init

enqueueLog

getQueues

addHFileRefs

run

sleepForRetries

isPeerEnabled

startup

terminate

terminate

terminate

getPeerClusterZnode

getPeerClusterId

getCurrentPath

getLastLoggedPath

getLastLoggedPosition

isSourceActive

getStats

getSourceMetrics

getWalGroupStatus