org.apache.mahout.h2obindings

Class H2OHelper

java.lang.Object

org.apache.mahout.h2obindings.H2OHelper

public class H2OHelper
extends Object

Collection of helper methods for H2O backend.

Constructor Summary

Constructors

Constructor and Description
H2OHelper()

Method Summary

Methods

Modifier and Type	Method and Description
static Matrix	allreduceBlock(H2ODrm drmA, Object bmfn, Object rfn)
static Vector	colMeans(water.fvec.Frame frame) Calculate Means of elements in a column, and return as a Vector.
static Vector	colSums(water.fvec.Frame frame) Calculate Sums of elements in a column, and return as a Vector.
static H2ODrm	drmFromMatrix(Matrix m, int minHint, int exactHint) Ingest a Mahout Matrix into an H2O DRM.
static H2ODrm	emptyDrm(long nrow, int ncol, int minHint, int exactHint) Create an empty (zero-filled) H2O DRM.
static water.fvec.Frame	emptyFrame(long nrow, int ncol, int minHint, int exactHint) Create an empty (zero-filled) H2O Frame efficiently.
static water.fvec.Frame	emptyFrame(long nrow, int ncol, int minHint, int exactHint, water.fvec.Vec.VectorGroup vg) Create an empty (zero-filled) H2O Frame efficiently.
static boolean	isSparse(water.fvec.Frame frame) Predicate to check if data is sparse in Frame.
static water.fvec.Vec	makeEmptyStrVec(water.fvec.Vec template) The following two methods: vecChunkLen and makeEmptyStrVec are h2o-0.1.25 specific.
static Matrix	matrixFromDrm(H2ODrm drm) Create a Mahout Matrix from a DRM.
static Vector	nonZeroCnt(water.fvec.Frame frame) Count non-zero elements in all columns, and return as a Vector.
static double	sumSqr(water.fvec.Frame frame) Calculate Sum of squares of all elements in the DRM.
static int	vecChunkLen(water.fvec.Vec template, int chunk)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

H2OHelper

public H2OHelper()

Method Detail

isSparse

public static boolean isSparse(water.fvec.Frame frame)

Predicate to check if data is sparse in Frame. If the number of missing elements is 32x times the number of present elements, consider it as sparse.

Parameters:

frame - Frame storing matrix data.

Returns:

True if data is sparse in Frame.

matrixFromDrm

public static Matrix matrixFromDrm(H2ODrm drm)

Create a Mahout Matrix from a DRM. Create either Sparse or Dense Matrix depending on number of missing elements in DRM.

Parameters:

drm - DRM object to create Matrix from.

Returns:

created Matrix.

colMeans

public static Vector colMeans(water.fvec.Frame frame)

Calculate Means of elements in a column, and return as a Vector. H2O precalculates means in a Vec, and a Vec corresponds to a column.

Parameters:

frame - Frame backing the H2O DRM.

Returns:

Vector of pre-calculated means.

colSums

public static Vector colSums(water.fvec.Frame frame)

Calculate Sums of elements in a column, and return as a Vector. Run an MRTask Job to add up sums. WARNING: Vulnerable to overflow. No way around it.

Parameters:

frame - Frame backing the H2O DRM.

Returns:

Vector of calculated sums.

sumSqr

public static double sumSqr(water.fvec.Frame frame)

Calculate Sum of squares of all elements in the DRM. Run an MRTask Job to add up sums of squares. WARNING: Vulnerable to overflow. No way around it.

Parameters:

frame - Frame backing the H2O DRM.

Returns:

Sum of squares of all elements in the DRM.

nonZeroCnt

public static Vector nonZeroCnt(water.fvec.Frame frame)

Count non-zero elements in all columns, and return as a Vector. Run an MRTask Job to count non-zero elements per column.

Parameters:

frame - Frame backing the H2O DRM.

Returns:

Vector of counted non-zero elements.

drmFromMatrix

public static H2ODrm drmFromMatrix(Matrix m,
                   int minHint,
                   int exactHint)

Ingest a Mahout Matrix into an H2O DRM. Frame is the backing data structure behind CheckpointedDrm.

Parameters:

m - Mahout Matrix to ingest data from.

minHint - Hint for minimum number of partitions in created DRM.

exactHint - Hint for exact number of partitions in created DRM.

Returns:

Created H2O backed DRM.

emptyFrame

public static water.fvec.Frame emptyFrame(long nrow,
                          int ncol,
                          int minHint,
                          int exactHint)

Create an empty (zero-filled) H2O Frame efficiently. Create a zero filled Frame with specified cardinality. Do not actually fill zeroes in each cell, create pre-compressed chunks. Time taken per column asymptotically at O(nChunks), not O(nrow).

Parameters:

nrow - Number of rows in the Frame.

ncol - Number of columns in the Frame.

minHint - Hint for minimum number of chunks per column in created Frame.

exactHint - Hint for exact number of chunks per column in created Frame.

Returns:

Created Frame.

emptyFrame

public static water.fvec.Frame emptyFrame(long nrow,
                          int ncol,
                          int minHint,
                          int exactHint,
                          water.fvec.Vec.VectorGroup vg)

Create an empty (zero-filled) H2O Frame efficiently. Create a zero filled Frame with specified cardinality. Do not actually fill zeroes in each cell, create pre-compressed chunks. Time taken per column asymptotically at O(nChunks), not O(nrow).

Parameters:

nrow - Number of rows in the Frame.

ncol - Number of columns in the Frame.

minHint - Hint for minimum number of chunks per column in created Frame.

exactHint - Hint for exact number of chunks per column in created Frame.

vg - Shared VectorGroup so that all columns are similarly partitioned.

Returns:

Created Frame.

makeEmptyStrVec

public static water.fvec.Vec makeEmptyStrVec(water.fvec.Vec template)

The following two methods: vecChunkLen and makeEmptyStrVec are h2o-0.1.25 specific.

vecChunkLen

public static int vecChunkLen(water.fvec.Vec template,
              int chunk)

emptyDrm

public static H2ODrm emptyDrm(long nrow,
              int ncol,
              int minHint,
              int exactHint)

Create an empty (zero-filled) H2O DRM. Create a zero filled DRM with specified cardinality. Use the efficient emptyFrame() method internally.

Parameters:

nrow - Number of rows in the Frame.

ncol - Number of columns in the Frame.

minHint - Hint for minimum number of chunks per column in created Frame.

exactHint - Hint for exact number of chunks per column in created Frame.

Returns:

Created DRM.

allreduceBlock

public static Matrix allreduceBlock(H2ODrm drmA,
                    Object bmfn,
                    Object rfn)