HFileOutputFormat2 (Apache HBase 1.7.2 API)

java.lang.Object
- org.apache.hadoop.mapreduce.OutputFormat<K,V>
- - org.apache.hadoop.mapreduce.lib.output.FileOutputFormat<ImmutableBytesWritable,Cell>
  - - org.apache.hadoop.hbase.mapreduce.HFileOutputFormat2

```
@InterfaceAudience.Public
@InterfaceStability.Evolving
public class HFileOutputFormat2
extends org.apache.hadoop.mapreduce.lib.output.FileOutputFormat<ImmutableBytesWritable,Cell>
```
Writes HFiles. Passed Cells must arrive in order. Writes current time as the sequence id for the file. Sets the major compacted attribute on created @{link HFiles. Calling write(null,null) will forcibly roll all HFiles being written.
Using this class as part of a MapReduce job is best done using configureIncrementalLoad(Job, Table, RegionLocator).

Field Summary

Fields
Modifier and Type	Field and Description
`static String`	`COMPRESSION_OVERRIDE_CONF_KEY`
`static String`	`DATABLOCK_ENCODING_OVERRIDE_CONF_KEY`
`static String`	`LOCALITY_SENSITIVE_CONF_KEY` Keep locality while generating HFiles for bulkload.
`static String`	`OUTPUT_TABLE_NAME_CONF_KEY`
`static String`	`REMOTE_CLUSTER_ZOOKEEPER_CLIENT_PORT_CONF_KEY`
`static String`	`REMOTE_CLUSTER_ZOOKEEPER_QUORUM_CONF_KEY`
`static String`	`REMOTE_CLUSTER_ZOOKEEPER_ZNODE_PARENT_CONF_KEY`
`static String`	`STORAGE_POLICY_PROPERTY`
`static String`	`STORAGE_POLICY_PROPERTY_CF_PREFIX`

Fields inherited from class org.apache.hadoop.mapreduce.lib.output.FileOutputFormat
BASE_OUTPUT_NAME, COMPRESS, COMPRESS_CODEC, COMPRESS_TYPE, OUTDIR, PART

Constructor Summary

Constructors
Constructor and Description

HFileOutputFormat2()

Constructors
Constructor and Description
`HFileOutputFormat2()`

Method Summary

Methods
Modifier and Type	Method and Description
`static void`	`configureIncrementalLoad(org.apache.hadoop.mapreduce.Job job, org.apache.hadoop.hbase.client.HTable table)` Deprecated. Use `configureIncrementalLoad(Job, Table, RegionLocator)` instead.
`static void`	`configureIncrementalLoad(org.apache.hadoop.mapreduce.Job job, HTableDescriptor tableDescriptor, RegionLocator regionLocator)` Configure a MapReduce Job to perform an incremental load into the given table.
`static void`	`configureIncrementalLoad(org.apache.hadoop.mapreduce.Job job, Table table, RegionLocator regionLocator)` Configure a MapReduce Job to perform an incremental load into the given table.
`static void`	`configureIncrementalLoadMap(org.apache.hadoop.mapreduce.Job job, Table table)`
`static void`	`configureRemoteCluster(org.apache.hadoop.mapreduce.Job job, org.apache.hadoop.conf.Configuration clusterConf)` Configure HBase cluster key for remote cluster to load region location for locality-sensitive if it's enabled.
`org.apache.hadoop.mapreduce.RecordWriter<ImmutableBytesWritable,Cell>`	`getRecordWriter(org.apache.hadoop.mapreduce.TaskAttemptContext context)`

Methods inherited from class org.apache.hadoop.mapreduce.lib.output.FileOutputFormat
checkOutputSpecs, getCompressOutput, getDefaultWorkFile, getOutputCommitter, getOutputCompressorClass, getOutputName, getOutputPath, getPathForWorkFile, getUniqueFile, getWorkOutputPath, setCompressOutput, setOutputCompressorClass, setOutputName, setOutputPath

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Field Detail
  - DATABLOCK_ENCODING_OVERRIDE_CONF_KEY
```
public static final String DATABLOCK_ENCODING_OVERRIDE_CONF_KEY
```
    See Also:
    Constant Field Values
  - COMPRESSION_OVERRIDE_CONF_KEY
```
public static final String COMPRESSION_OVERRIDE_CONF_KEY
```
    See Also:
    Constant Field Values
  - LOCALITY_SENSITIVE_CONF_KEY
```
public static final String LOCALITY_SENSITIVE_CONF_KEY
```
    Keep locality while generating HFiles for bulkload. See HBASE-12596
    
    See Also:
    Constant Field Values
  - OUTPUT_TABLE_NAME_CONF_KEY
```
public static final String OUTPUT_TABLE_NAME_CONF_KEY
```
    See Also:
    Constant Field Values
  - REMOTE_CLUSTER_ZOOKEEPER_QUORUM_CONF_KEY
```
public static final String REMOTE_CLUSTER_ZOOKEEPER_QUORUM_CONF_KEY
```
    See Also:
    Constant Field Values
  - REMOTE_CLUSTER_ZOOKEEPER_CLIENT_PORT_CONF_KEY
```
public static final String REMOTE_CLUSTER_ZOOKEEPER_CLIENT_PORT_CONF_KEY
```
    See Also:
    Constant Field Values
  - REMOTE_CLUSTER_ZOOKEEPER_ZNODE_PARENT_CONF_KEY
```
public static final String REMOTE_CLUSTER_ZOOKEEPER_ZNODE_PARENT_CONF_KEY
```
    See Also:
    Constant Field Values
  - STORAGE_POLICY_PROPERTY
```
public static final String STORAGE_POLICY_PROPERTY
```
    See Also:
    Constant Field Values
  - STORAGE_POLICY_PROPERTY_CF_PREFIX
```
public static final String STORAGE_POLICY_PROPERTY_CF_PREFIX
```
    See Also:
    Constant Field Values
- Constructor Detail
  - HFileOutputFormat2
```
public HFileOutputFormat2()
```
- Method Detail
  - getRecordWriter
```
public org.apache.hadoop.mapreduce.RecordWriter<ImmutableBytesWritable,Cell> getRecordWriter(org.apache.hadoop.mapreduce.TaskAttemptContext context)
                                                                                      throws IOException,
                                                                                             InterruptedException
```
    Specified by:
    
    getRecordWriter in class org.apache.hadoop.mapreduce.lib.output.FileOutputFormat<ImmutableBytesWritable,Cell>
    
    Throws:
    
    IOException
    
    InterruptedException
  - configureIncrementalLoad
```
@Deprecated
public static void configureIncrementalLoad(org.apache.hadoop.mapreduce.Job job,
                                       org.apache.hadoop.hbase.client.HTable table)
                                     throws IOException
```
    Deprecated. Use configureIncrementalLoad(Job, Table, RegionLocator) instead.
    Configure a MapReduce Job to perform an incremental load into the given table. This
    - Inspects the table to configure a total order partitioner
    - Uploads the partitions file to the cluster and adds it to the DistributedCache
    - Sets the number of reduce tasks to match the current number of regions
    - Sets the output key/value class to match HFileOutputFormat2's requirements
    - Sets the reducer up to perform the appropriate sorting (either KeyValueSortReducer or PutSortReducer)
    The user should be sure to set the map output value class to either KeyValue or Put before running this function.
    Throws:
    
    IOException
  - configureIncrementalLoad
```
public static void configureIncrementalLoad(org.apache.hadoop.mapreduce.Job job,
                            Table table,
                            RegionLocator regionLocator)
                                     throws IOException
```
    Configure a MapReduce Job to perform an incremental load into the given table. This
    - Inspects the table to configure a total order partitioner
    - Uploads the partitions file to the cluster and adds it to the DistributedCache
    - Sets the number of reduce tasks to match the current number of regions
    - Sets the output key/value class to match HFileOutputFormat2's requirements
    - Sets the reducer up to perform the appropriate sorting (either KeyValueSortReducer or PutSortReducer)
    - Sets the HBase cluster key to load region locations for locality-sensitive
    The user should be sure to set the map output value class to either KeyValue or Put before running this function.
    Throws:
    
    IOException
  - configureIncrementalLoad
```
public static void configureIncrementalLoad(org.apache.hadoop.mapreduce.Job job,
                            HTableDescriptor tableDescriptor,
                            RegionLocator regionLocator)
                                     throws IOException
```
    Configure a MapReduce Job to perform an incremental load into the given table. This
    - Inspects the table to configure a total order partitioner
    - Uploads the partitions file to the cluster and adds it to the DistributedCache
    - Sets the number of reduce tasks to match the current number of regions
    - Sets the output key/value class to match HFileOutputFormat2's requirements
    - Sets the reducer up to perform the appropriate sorting (either KeyValueSortReducer or PutSortReducer)
    The user should be sure to set the map output value class to either KeyValue or Put before running this function.
    Throws:
    
    IOException
  - configureIncrementalLoadMap
```
public static void configureIncrementalLoadMap(org.apache.hadoop.mapreduce.Job job,
                               Table table)
                                        throws IOException
```
    Throws:
    
    IOException
  - configureRemoteCluster
```
public static void configureRemoteCluster(org.apache.hadoop.mapreduce.Job job,
                          org.apache.hadoop.conf.Configuration clusterConf)
```
    Configure HBase cluster key for remote cluster to load region location for locality-sensitive if it's enabled. It's not necessary to call this method explicitly when the cluster key for HBase cluster to be used to load region location is configured in the job configuration. Call this method when another HBase cluster key is configured in the job configuration. For example, you should call when you load data from HBase cluster A using TableInputFormat and generate hfiles for HBase cluster B. Otherwise, HFileOutputFormat2 fetch location from cluster A and locality-sensitive won't working correctly. configureIncrementalLoad(Job, Table, RegionLocator) calls this method using Table.getConfiguration() as clusterConf. See HBASE-25608.
    
    Parameters:
    job - which has configuration to be updated
    clusterConf - which contains cluster key of the HBase cluster to be locality-sensitive
    See Also:
    configureIncrementalLoad(Job, Table, RegionLocator), LOCALITY_SENSITIVE_CONF_KEY, REMOTE_CLUSTER_ZOOKEEPER_QUORUM_CONF_KEY, REMOTE_CLUSTER_ZOOKEEPER_CLIENT_PORT_CONF_KEY, REMOTE_CLUSTER_ZOOKEEPER_ZNODE_PARENT_CONF_KEY

Class HFileOutputFormat2

Field Summary

Fields inherited from class org.apache.hadoop.mapreduce.lib.output.FileOutputFormat

Constructor Summary

Method Summary

Methods inherited from class org.apache.hadoop.mapreduce.lib.output.FileOutputFormat

Methods inherited from class java.lang.Object

Field Detail

DATABLOCK_ENCODING_OVERRIDE_CONF_KEY

COMPRESSION_OVERRIDE_CONF_KEY

LOCALITY_SENSITIVE_CONF_KEY

OUTPUT_TABLE_NAME_CONF_KEY

REMOTE_CLUSTER_ZOOKEEPER_QUORUM_CONF_KEY

REMOTE_CLUSTER_ZOOKEEPER_CLIENT_PORT_CONF_KEY

REMOTE_CLUSTER_ZOOKEEPER_ZNODE_PARENT_CONF_KEY

STORAGE_POLICY_PROPERTY

STORAGE_POLICY_PROPERTY_CF_PREFIX

Constructor Detail

HFileOutputFormat2

Method Detail

getRecordWriter

configureIncrementalLoad

configureIncrementalLoad

configureIncrementalLoad

configureIncrementalLoadMap

configureRemoteCluster