Package org.apache.heron.api
Class Config
- java.lang.Object
-
- java.util.AbstractMap<K,V>
-
- java.util.HashMap<String,Object>
-
- org.apache.heron.api.Config
-
- All Implemented Interfaces:
Serializable,Cloneable,Map<String,Object>
public class Config extends HashMap<String,Object>
Topology configs are specified as a plain old map. This class provides a convenient way to create a topology config map by providing setter methods for all the configs that can be set. It also makes it easier to do things like add serializations.Note that you may put other configurations in any of the configs. Heron will ignore anything it doesn't recognize, but your topologies are free to make use of them by reading them in the prepare method of Bolts or the open method of Spouts.
- See Also:
- Serialized Form
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static classConfig.TopologyReliabilityModeWhat is the reliability mode under which we are running this topology Topology writers must set TOPOLOGY_RELIABILITY_MODE to one one of the following modes-
Nested classes/interfaces inherited from class java.util.AbstractMap
AbstractMap.SimpleEntry<K extends Object,V extends Object>, AbstractMap.SimpleImmutableEntry<K extends Object,V extends Object>
-
-
Field Summary
Fields Modifier and Type Field Description static StringHERON_JAVA_SERIALIZER_CLASS_NAMEThe serializers available for TOPOLOGY_SERIALIZER_CLASSNAME.static StringHERON_KRYO_SERIALIZER_CLASS_NAMEstatic StringTOPOLOGY_ADDITIONAL_CLASSPATHAny user defined classpath that needs to be passed to instances should be set in to config through this key.static StringTOPOLOGY_AUTO_TASK_HOOKSA list of task hooks that are automatically added to every spout and bolt in the topology.static StringTOPOLOGY_CAP_TICKETCap ticket (if filed) for the topology.static StringTOPOLOGY_COMPONENT_CPUMAPPer component RAM requirement.static StringTOPOLOGY_COMPONENT_DISKMAPPer component RAM requirement.static StringTOPOLOGY_COMPONENT_JVMOPTSPer component JVM options.static StringTOPOLOGY_COMPONENT_OUTPUT_BPSThe per component output bytes per second (rate limit) in this topology.static StringTOPOLOGY_COMPONENT_PARALLELISMThe per component parallelism for a component in this topology.static StringTOPOLOGY_COMPONENT_RAMMAPPer component RAM requirement.static StringTOPOLOGY_CONTAINER_CPU_PADDINGAmount of CPU to pad each container.static StringTOPOLOGY_CONTAINER_CPU_REQUESTEDNumber of CPU cores per container to be reserved for this topologystatic StringTOPOLOGY_CONTAINER_DISK_REQUESTEDAmount of disk per container to be reserved for this topology.static StringTOPOLOGY_CONTAINER_MAX_CPU_HINTHint for max number of CPU cores per container to be reserved for this topologystatic StringTOPOLOGY_CONTAINER_MAX_DISK_HINTHint for max amount of disk per container to be reserved for this topology.static StringTOPOLOGY_CONTAINER_MAX_NUM_INSTANCESMax number of instances per container for this topology.static StringTOPOLOGY_CONTAINER_MAX_RAM_HINTHint for max amount of RAM per container to be reserved for this topology.static StringTOPOLOGY_CONTAINER_PADDING_PERCENTAGEPercent of resource to pad each container.static StringTOPOLOGY_CONTAINER_RAM_PADDINGAmount of RAM to pad each container.static StringTOPOLOGY_CONTAINER_RAM_REQUESTEDAmount of RAM per container to be reserved for this topology.static StringTOPOLOGY_DEBUGWhen set to true, Heron will log every message that's emitted.static StringTOPOLOGY_DROPTUPLES_UPON_BACKPRESSUREDo we want to drop tuples instead of initiating Spout BackPressurestatic StringTOPOLOGY_ENABLE_ACKINGDeprecated.useTOPOLOGY_RELIABILITY_MODEinstead.static StringTOPOLOGY_ENABLE_MESSAGE_TIMEOUTSTrue if Heron should timeout messages or not.static StringTOPOLOGY_ENVIRONMENTTopology-specific environment properties to be added to an Heron instance.static StringTOPOLOGY_FALL_BACK_ON_JAVA_SERIALIZATIONWhether or not to fallback to Java serialization in a topology.static StringTOPOLOGY_KRYO_DECORATORSA list of classes that customize storm's kryo instance during start-up.static StringTOPOLOGY_KRYO_FACTORYClass that specifies how to create a Kryo instance for serialization.static StringTOPOLOGY_KRYO_REGISTERA list of serialization registrations if KryoSerializor is used.static StringTOPOLOGY_MAX_SPOUT_PENDINGThe maximum number of tuples that can be pending on a spout task at any given time.static StringTOPOLOGY_MESSAGE_TIMEOUT_SECSThe maximum amount of time given to the topology to fully process a message emitted by a spout.static StringTOPOLOGY_METRICSMGR_RAMMetrics Manager RAM requirementstatic StringTOPOLOGY_NAMEName of the topology.static StringTOPOLOGY_PACKING_FFD_SORTING_STRATEGYSorting strategy for FirstFitDecreasingPacking algorithm.static StringTOPOLOGY_PROJECT_NAMEProject name of the topology, to help us with tagging which topologies are part of which project.static StringTOPOLOGY_RELIABILITY_MODEA Heron topology can be run in any one of the TopologyReliabilityMode mode.static StringTOPOLOGY_REMOTE_DEBUGGING_ENABLEEnable Remote debugging for java heron instancesstatic StringTOPOLOGY_SERIALIZER_CLASSNAMEThe serialization class that is used to serialize/deserialize tuplesstatic StringTOPOLOGY_SKIP_MISSING_KRYO_REGISTRATIONSWhether or not Heron should skip the loading of kryo registrations for which it does not know the class or have the serializer implementation.static StringTOPOLOGY_STATEFUL_CHECKPOINT_INTERVAL_SECONDSWhat's the checkpoint interval for stateful topologies in secondsstatic StringTOPOLOGY_STATEFUL_CKPTMGR_RAMCheckpoint Manager RAM requirement.static StringTOPOLOGY_STATEFUL_SPILL_STATEWhether spill the state to disk for transferringstatic StringTOPOLOGY_STATEFUL_SPILL_STATE_LOCATIONThe local disk location where the state is spilledstatic StringTOPOLOGY_STATEFUL_START_CLEANBoolean flag that says that the stateful topology should start from clean state, i.e.static StringTOPOLOGY_STMGR_RAMStream Manager RAM requirementstatic StringTOPOLOGY_STMGRSThe number of stmgr instances that should spin up to service this topology.static StringTOPOLOGY_TEAM_EMAILEmail of the team which owns this topology.static StringTOPOLOGY_TEAM_ENVIRONMENTName of the of the environment this topology should run in.static StringTOPOLOGY_TEAM_NAMEName of the team which owns this topology.static StringTOPOLOGY_TICK_TUPLE_FREQ_MSHow often (in milliseconds) a tick tuple from the "__system" component and "__tick" stream should be sent to tasks.static StringTOPOLOGY_TIMER_EVENTSTimer events registered for a topology.static StringTOPOLOGY_UPDATE_DEACTIVATE_WAIT_SECSAmount of time to wait after deactivating a topology before updating itstatic StringTOPOLOGY_UPDATE_REACTIVATE_WAIT_SECSAfter updating a topology, amount of time to wait for it to come back up before reactivating itstatic StringTOPOLOGY_WORKER_CHILDOPTSTopology-specific options for the worker child process.
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Deprecated Methods Modifier and Type Method Description voidaddClasspath(Map<String,Object> conf, String classpath)Set<String>getApiVars()List<String>getAutoTaskHooks()static List<String>getAutoTaskHooks(Map<String,Object> conf)voidregisterDecorator(Class<? extends IKryoDecorator> klass)static voidregisterDecorator(Map conf, Class<? extends IKryoDecorator> klass)voidregisterKryoSerialization(Class klass)voidregisterKryoSerialization(Class klass, Class<? extends com.esotericsoftware.kryo.Serializer> serializerClass)static voidregisterKryoSerialization(Map<String,Object> conf, Class klass)static voidregisterKryoSerialization(Map<String,Object> conf, Class klass, Class<? extends com.esotericsoftware.kryo.Serializer> serializerClass)voidregisterTopologyTimerEvents(String name, Duration interval, Runnable task)static voidregisterTopologyTimerEvents(Map<String,Object> conf, String name, Duration interval, Runnable task)Registers a timer event that executes periodicallyvoidsetAutoTaskHooks(List<String> hooks)static voidsetAutoTaskHooks(Map<String,Object> conf, List<String> hooks)static voidsetCheckpointManagerRam(Map<String,Object> conf, org.apache.heron.common.basics.ByteAmount ramInBytes)voidsetCheckpointManagerRam(org.apache.heron.common.basics.ByteAmount ramInBytes)voidsetComponentCpu(String component, double cpu)static voidsetComponentCpu(Map<String,Object> conf, String component, double cpu)voidsetComponentCpuMap(String cpuMap)static voidsetComponentCpuMap(Map<String,Object> conf, String cpuMap)voidsetComponentDisk(String component, org.apache.heron.common.basics.ByteAmount diskInBytes)static voidsetComponentDisk(Map<String,Object> conf, String component, org.apache.heron.common.basics.ByteAmount diskInBytes)voidsetComponentDiskMap(String diskMap)static voidsetComponentDiskMap(Map<String,Object> conf, String diskMap)voidsetComponentJvmOptions(String component, String jvmOptions)static voidsetComponentJvmOptions(Map<String,Object> conf, String component, String jvmOptions)voidsetComponentParallelism(int parallelism)static voidsetComponentParallelism(Map<String,Object> conf, int parallelism)voidsetComponentRam(String component, org.apache.heron.common.basics.ByteAmount ramInBytes)static voidsetComponentRam(Map<String,Object> conf, String component, long ramInBytes)Deprecated.use setComponentRam(Map<String, Object> conf, String component, ByteAmount ramInBytes)static voidsetComponentRam(Map<String,Object> conf, String component, org.apache.heron.common.basics.ByteAmount ramInBytes)voidsetComponentRamMap(String ramMap)static voidsetComponentRamMap(Map<String,Object> conf, String ramMap)voidsetContainerCpuPadding(double ncpus)static voidsetContainerCpuPadding(Map<String,Object> conf, double ncpus)voidsetContainerCpuRequested(double ncpus)static voidsetContainerCpuRequested(Map<String,Object> conf, double ncpus)static voidsetContainerDiskRequested(Map<String,Object> conf, long nbytes)Deprecated.use setContainerDiskRequested(Map<String, Object> conf, ByteAmount nbytes)static voidsetContainerDiskRequested(Map<String,Object> conf, org.apache.heron.common.basics.ByteAmount nbytes)voidsetContainerDiskRequested(org.apache.heron.common.basics.ByteAmount nbytes)voidsetContainerMaxCpuHint(double ncpus)static voidsetContainerMaxCpuHint(Map<String,Object> conf, double ncpus)static voidsetContainerMaxDiskHint(Map<String,Object> conf, org.apache.heron.common.basics.ByteAmount nbytes)voidsetContainerMaxDiskHint(org.apache.heron.common.basics.ByteAmount nbytes)static voidsetContainerMaxRamHint(Map<String,Object> conf, org.apache.heron.common.basics.ByteAmount nbytes)voidsetContainerMaxRamHint(org.apache.heron.common.basics.ByteAmount nbytes)voidsetContainerPaddingPercentage(int percentage)static voidsetContainerPaddingPercentage(Map<String,Object> conf, int percentage)static voidsetContainerRamPadding(Map<String,Object> conf, org.apache.heron.common.basics.ByteAmount nbytes)voidsetContainerRamPadding(org.apache.heron.common.basics.ByteAmount nbytes)static voidsetContainerRamRequested(Map<String,Object> conf, long nbytes)Deprecated.use setContainerRamRequested(Map<String, Object> conf, ByteAmount nbytes)static voidsetContainerRamRequested(Map<String,Object> conf, org.apache.heron.common.basics.ByteAmount nbytes)voidsetContainerRamRequested(org.apache.heron.common.basics.ByteAmount nbytes)voidsetDebug(boolean isOn)static voidsetDebug(Map<String,Object> conf, boolean isOn)voidsetEnableAcking(boolean acking)Deprecated.static voidsetEnableAcking(Map<String,Object> conf, boolean acking)Deprecated.static voidsetEnvironment(Map<String,Object> conf, Map env)voidsetFallBackOnJavaSerialization(boolean fallback)static voidsetFallBackOnJavaSerialization(Map conf, boolean fallback)voidsetFFDSortingStrategy(String sortingStrategy)static voidsetFFDSortingStrategy(Map<String,Object> conf, String sortingStrategy)voidsetKryoFactory(Class<? extends IKryoFactory> klass)static voidsetKryoFactory(Map conf, Class<? extends IKryoFactory> klass)static voidsetMaxNumInstancesPerContainer(Map<String,Object> conf, int n)voidsetMaxSpoutPending(int max)static voidsetMaxSpoutPending(Map<String,Object> conf, int max)voidsetMessageTimeoutSecs(int secs)static voidsetMessageTimeoutSecs(Map<String,Object> conf, int secs)static voidsetMetricsmgrRam(Map<String,Object> conf, org.apache.heron.common.basics.ByteAmount ramInBytes)voidsetMetricsmgrRam(org.apache.heron.common.basics.ByteAmount ramInBytes)voidsetNumStmgrs(int stmgrs)static voidsetNumStmgrs(Map<String,Object> conf, int stmgrs)voidsetSerializationClassName(String className)static voidsetSerializationClassName(Map<String,Object> conf, String className)voidsetSkipMissingKryoRegistrations(boolean skip)static voidsetSkipMissingKryoRegistrations(Map conf, boolean skip)static voidsetStreamManagerRam(Map<String,Object> conf, org.apache.heron.common.basics.ByteAmount ramInBytes)voidsetStreamManagerRam(org.apache.heron.common.basics.ByteAmount ramInBytes)voidsetTeamEmail(String teamEmail)static voidsetTeamEmail(Map<String,Object> conf, String teamEmail)voidsetTeamName(String teamName)static voidsetTeamName(Map<String,Object> conf, String teamName)voidsetTickTupleFrequency(int seconds)static voidsetTickTupleFrequency(Map<String,Object> conf, int seconds)static voidsetTickTupleFrequencyMs(Map<String,Object> conf, long millis)voidsetTopologyCapTicket(String ticket)static voidsetTopologyCapTicket(Map<String,Object> conf, String ticket)voidsetTopologyComponentOutputBPS(long bps)Deprecated.use the TOPOLOGY_COMPONENT_OUTPUT_BPS config with ComponentConfigurationDeclarer's addConfiguration() instead.static voidsetTopologyComponentOutputBPS(Map<String,Object> conf, long bps)Deprecated.use the TOPOLOGY_COMPONENT_OUTPUT_BPS config with ComponentConfigurationDeclarer's addConfiguration() instead.voidsetTopologyContainerMaxNumInstances(int n)voidsetTopologyDropTuplesUponBackpressure(boolean dropTuples)voidsetTopologyProjectName(String project)static voidsetTopologyProjectName(Map<String,Object> conf, String project)static voidsetTopologyReliabilityMode(Map<String,Object> conf, Config.TopologyReliabilityMode mode)voidsetTopologyReliabilityMode(Config.TopologyReliabilityMode mode)voidsetTopologyRemoteDebugging(boolean isOn)voidsetTopologyStatefulCheckpointIntervalSecs(int secs)static voidsetTopologyStatefulCheckpointIntervalSecs(Map<String,Object> conf, int secs)voidsetTopologyStatefulSpillState(String spillState)static voidsetTopologyStatefulSpillState(Map<String,Object> conf, String spillState)voidsetTopologyStatefulSpillStateLocation(String location)static voidsetTopologyStatefulSpillStateLocation(Map<String,Object> conf, String location)voidsetTopologyStatefulStartClean(boolean clean)static voidsetTopologyStatefulStartClean(Map<String,Object> conf, boolean clean)voidsetUpdateDeactivateWaitDuration(int seconds)voidsetUpdateReactivateWaitDuration(int seconds)-
Methods inherited from class java.util.HashMap
clear, clone, compute, computeIfAbsent, computeIfPresent, containsKey, containsValue, entrySet, forEach, get, getOrDefault, isEmpty, keySet, merge, put, putAll, putIfAbsent, remove, remove, replace, replace, replaceAll, size, values
-
Methods inherited from class java.util.AbstractMap
equals, hashCode, toString
-
-
-
-
Field Detail
-
TOPOLOGY_WORKER_CHILDOPTS
public static final String TOPOLOGY_WORKER_CHILDOPTS
Topology-specific options for the worker child process. This is used in addition to WORKER_CHILDOPTS.- See Also:
- Constant Field Values
-
TOPOLOGY_COMPONENT_JVMOPTS
public static final String TOPOLOGY_COMPONENT_JVMOPTS
Per component JVM options. The format of this flag is something like spout0:jvmopt_for_spout0,spout1:jvmopt_for_spout1. Mostly should be used in conjunction with setComponentJvmOptions(). This is used in addition to TOPOLOGY_WORKER_CHILDOPTS. While TOPOLOGY_WORKER_CHILDOPTS applies for all components, this is per component- See Also:
- Constant Field Values
-
TOPOLOGY_TICK_TUPLE_FREQ_MS
public static final String TOPOLOGY_TICK_TUPLE_FREQ_MS
How often (in milliseconds) a tick tuple from the "__system" component and "__tick" stream should be sent to tasks. Meant to be used as a component-specific configuration.- See Also:
- Constant Field Values
-
TOPOLOGY_ENABLE_MESSAGE_TIMEOUTS
public static final String TOPOLOGY_ENABLE_MESSAGE_TIMEOUTS
True if Heron should timeout messages or not. Defaults to true. This is meant to be used in unit tests to prevent tuples from being accidentally timed out during the test.- See Also:
- Constant Field Values
-
TOPOLOGY_DEBUG
public static final String TOPOLOGY_DEBUG
When set to true, Heron will log every message that's emitted.- See Also:
- Constant Field Values
-
TOPOLOGY_STMGRS
public static final String TOPOLOGY_STMGRS
The number of stmgr instances that should spin up to service this topology. All the executors will be evenly shared by these stmgrs.- See Also:
- Constant Field Values
-
TOPOLOGY_MESSAGE_TIMEOUT_SECS
public static final String TOPOLOGY_MESSAGE_TIMEOUT_SECS
The maximum amount of time given to the topology to fully process a message emitted by a spout. If the message is not acked within this time frame, Heron will fail the message on the spout. Some spouts implementations will then replay the message at a later time.- See Also:
- Constant Field Values
-
TOPOLOGY_COMPONENT_PARALLELISM
public static final String TOPOLOGY_COMPONENT_PARALLELISM
The per component parallelism for a component in this topology. Note:- If you are changing this, please change the utils.h as well- See Also:
- Constant Field Values
-
TOPOLOGY_MAX_SPOUT_PENDING
public static final String TOPOLOGY_MAX_SPOUT_PENDING
The maximum number of tuples that can be pending on a spout task at any given time. This config applies to individual tasks, not to spouts or topologies as a whole.A pending tuple is one that has been emitted from a spout but has not been acked or failed yet. Note that this config parameter has no effect for unreliable spouts that don't tag their tuples with a message id.
- See Also:
- Constant Field Values
-
TOPOLOGY_AUTO_TASK_HOOKS
public static final String TOPOLOGY_AUTO_TASK_HOOKS
A list of task hooks that are automatically added to every spout and bolt in the topology. An example of when you'd do this is to add a hook that integrates with your internal monitoring system. These hooks are instantiated using the zero-arg constructor.- See Also:
- Constant Field Values
-
TOPOLOGY_SERIALIZER_CLASSNAME
public static final String TOPOLOGY_SERIALIZER_CLASSNAME
The serialization class that is used to serialize/deserialize tuples- See Also:
- Constant Field Values
-
HERON_JAVA_SERIALIZER_CLASS_NAME
public static final String HERON_JAVA_SERIALIZER_CLASS_NAME
The serializers available for TOPOLOGY_SERIALIZER_CLASSNAME.- See Also:
- Constant Field Values
-
HERON_KRYO_SERIALIZER_CLASS_NAME
public static final String HERON_KRYO_SERIALIZER_CLASS_NAME
- See Also:
- Constant Field Values
-
TOPOLOGY_KRYO_FACTORY
public static final String TOPOLOGY_KRYO_FACTORY
Class that specifies how to create a Kryo instance for serialization. Heron will then apply topology.kryo.register. The default implementation implements topology.fall.back.on.java.serialization and turns references off.- See Also:
- Constant Field Values
-
TOPOLOGY_KRYO_REGISTER
public static final String TOPOLOGY_KRYO_REGISTER
A list of serialization registrations if KryoSerializor is used. In Kryo, the serialization can be the name of a class (in which case Kryo will automatically create a serializer for the class that saves all the object's fields), or an implementation of Kryo Serializer.See Kryo's documentation for more information about writing custom serializers.
- See Also:
- Constant Field Values
-
TOPOLOGY_KRYO_DECORATORS
public static final String TOPOLOGY_KRYO_DECORATORS
A list of classes that customize storm's kryo instance during start-up. Each listed class name must implement IKryoDecorator. During start-up the listed class is instantiated with 0 arguments, then its 'decorate' method is called with storm's kryo instance as the only argument.- See Also:
- Constant Field Values
-
TOPOLOGY_SKIP_MISSING_KRYO_REGISTRATIONS
public static final String TOPOLOGY_SKIP_MISSING_KRYO_REGISTRATIONS
Whether or not Heron should skip the loading of kryo registrations for which it does not know the class or have the serializer implementation. Otherwise, the task will fail to load and will throw an error at runtime. The use case of this is if you want to declare your serializations on the heron.yaml files on the cluster rather than every single time you submit a topology. Different applications may use different serializations and so a single application may not have the code for the other serializers used by other apps. By setting this config to true, Heron will ignore that it doesn't have those other serializations rather than throw an error.- See Also:
- Constant Field Values
-
TOPOLOGY_FALL_BACK_ON_JAVA_SERIALIZATION
public static final String TOPOLOGY_FALL_BACK_ON_JAVA_SERIALIZATION
Whether or not to fallback to Java serialization in a topology.- See Also:
- Constant Field Values
-
TOPOLOGY_ENABLE_ACKING
@Deprecated public static final String TOPOLOGY_ENABLE_ACKING
Deprecated.useTOPOLOGY_RELIABILITY_MODEinstead.Is the topology running in atleast-once mode?If this is set to false, then Heron will immediately ack tuples as soon as they come off the spout, effectively disabling reliability.
- See Also:
- Constant Field Values
-
TOPOLOGY_RELIABILITY_MODE
public static final String TOPOLOGY_RELIABILITY_MODE
A Heron topology can be run in any one of the TopologyReliabilityMode mode. The format of this flag is the string encoded values of the underlying TopologyReliabilityMode value.- See Also:
- Constant Field Values
-
TOPOLOGY_CONTAINER_CPU_REQUESTED
public static final String TOPOLOGY_CONTAINER_CPU_REQUESTED
Number of CPU cores per container to be reserved for this topology- See Also:
- Constant Field Values
-
TOPOLOGY_CONTAINER_RAM_REQUESTED
public static final String TOPOLOGY_CONTAINER_RAM_REQUESTED
Amount of RAM per container to be reserved for this topology. In bytes.- See Also:
- Constant Field Values
-
TOPOLOGY_CONTAINER_DISK_REQUESTED
public static final String TOPOLOGY_CONTAINER_DISK_REQUESTED
Amount of disk per container to be reserved for this topology. In bytes.- See Also:
- Constant Field Values
-
TOPOLOGY_CONTAINER_MAX_CPU_HINT
public static final String TOPOLOGY_CONTAINER_MAX_CPU_HINT
Hint for max number of CPU cores per container to be reserved for this topology- See Also:
- Constant Field Values
-
TOPOLOGY_CONTAINER_MAX_RAM_HINT
public static final String TOPOLOGY_CONTAINER_MAX_RAM_HINT
Hint for max amount of RAM per container to be reserved for this topology. In bytes.- See Also:
- Constant Field Values
-
TOPOLOGY_CONTAINER_MAX_DISK_HINT
public static final String TOPOLOGY_CONTAINER_MAX_DISK_HINT
Hint for max amount of disk per container to be reserved for this topology. In bytes.- See Also:
- Constant Field Values
-
TOPOLOGY_CONTAINER_MAX_NUM_INSTANCES
public static final String TOPOLOGY_CONTAINER_MAX_NUM_INSTANCES
Max number of instances per container for this topology.- See Also:
- Constant Field Values
-
TOPOLOGY_CONTAINER_PADDING_PERCENTAGE
public static final String TOPOLOGY_CONTAINER_PADDING_PERCENTAGE
Percent of resource to pad each container.- See Also:
- Constant Field Values
-
TOPOLOGY_CONTAINER_RAM_PADDING
public static final String TOPOLOGY_CONTAINER_RAM_PADDING
Amount of RAM to pad each container. In bytes.- See Also:
- Constant Field Values
-
TOPOLOGY_CONTAINER_CPU_PADDING
public static final String TOPOLOGY_CONTAINER_CPU_PADDING
Amount of CPU to pad each container. In decimal form of percent of CPU shares.- See Also:
- Constant Field Values
-
TOPOLOGY_COMPONENT_CPUMAP
public static final String TOPOLOGY_COMPONENT_CPUMAP
Per component RAM requirement. The format of this flag is something like spout0:0.2,spout1:0.2,bolt1:0.5.- See Also:
- Constant Field Values
-
TOPOLOGY_COMPONENT_RAMMAP
public static final String TOPOLOGY_COMPONENT_RAMMAP
Per component RAM requirement. The format of this flag is something like spout0:12434,spout1:345353,bolt1:545356.- See Also:
- Constant Field Values
-
TOPOLOGY_COMPONENT_DISKMAP
public static final String TOPOLOGY_COMPONENT_DISKMAP
Per component RAM requirement. The format of this flag is something like spout0:12434,spout1:345353,bolt1:545356.- See Also:
- Constant Field Values
-
TOPOLOGY_PACKING_FFD_SORTING_STRATEGY
public static final String TOPOLOGY_PACKING_FFD_SORTING_STRATEGY
Sorting strategy for FirstFitDecreasingPacking algorithm. RAM_FIRST (default), or CPU_FIRST- See Also:
- Constant Field Values
-
TOPOLOGY_STATEFUL_CHECKPOINT_INTERVAL_SECONDS
public static final String TOPOLOGY_STATEFUL_CHECKPOINT_INTERVAL_SECONDS
What's the checkpoint interval for stateful topologies in seconds- See Also:
- Constant Field Values
-
TOPOLOGY_STATEFUL_START_CLEAN
public static final String TOPOLOGY_STATEFUL_START_CLEAN
Boolean flag that says that the stateful topology should start from clean state, i.e. ignore any checkpoint state- See Also:
- Constant Field Values
-
TOPOLOGY_STATEFUL_CKPTMGR_RAM
public static final String TOPOLOGY_STATEFUL_CKPTMGR_RAM
Checkpoint Manager RAM requirement.- See Also:
- Constant Field Values
-
TOPOLOGY_STATEFUL_SPILL_STATE
public static final String TOPOLOGY_STATEFUL_SPILL_STATE
Whether spill the state to disk for transferring- See Also:
- Constant Field Values
-
TOPOLOGY_STATEFUL_SPILL_STATE_LOCATION
public static final String TOPOLOGY_STATEFUL_SPILL_STATE_LOCATION
The local disk location where the state is spilled- See Also:
- Constant Field Values
-
TOPOLOGY_STMGR_RAM
public static final String TOPOLOGY_STMGR_RAM
Stream Manager RAM requirement- See Also:
- Constant Field Values
-
TOPOLOGY_METRICSMGR_RAM
public static final String TOPOLOGY_METRICSMGR_RAM
Metrics Manager RAM requirement- See Also:
- Constant Field Values
-
TOPOLOGY_NAME
public static final String TOPOLOGY_NAME
Name of the topology. This config is automatically set by Heron when the topology is submitted.- See Also:
- Constant Field Values
-
TOPOLOGY_TEAM_NAME
public static final String TOPOLOGY_TEAM_NAME
Name of the team which owns this topology.- See Also:
- Constant Field Values
-
TOPOLOGY_TEAM_EMAIL
public static final String TOPOLOGY_TEAM_EMAIL
Email of the team which owns this topology.- See Also:
- Constant Field Values
-
TOPOLOGY_TEAM_ENVIRONMENT
public static final String TOPOLOGY_TEAM_ENVIRONMENT
Name of the of the environment this topology should run in.- See Also:
- Constant Field Values
-
TOPOLOGY_CAP_TICKET
public static final String TOPOLOGY_CAP_TICKET
Cap ticket (if filed) for the topology. If the topology is in prod this has to be set or it cannot be deployed.- See Also:
- Constant Field Values
-
TOPOLOGY_PROJECT_NAME
public static final String TOPOLOGY_PROJECT_NAME
Project name of the topology, to help us with tagging which topologies are part of which project. For example, if topology A and Topology B are part of the same project, we will like to aggregate them as part of the same project. This is required by Cap team.- See Also:
- Constant Field Values
-
TOPOLOGY_ADDITIONAL_CLASSPATH
public static final String TOPOLOGY_ADDITIONAL_CLASSPATH
Any user defined classpath that needs to be passed to instances should be set in to config through this key. The value will be of the format "cp1:cp2:cp3..."- See Also:
- Constant Field Values
-
TOPOLOGY_UPDATE_DEACTIVATE_WAIT_SECS
public static final String TOPOLOGY_UPDATE_DEACTIVATE_WAIT_SECS
Amount of time to wait after deactivating a topology before updating it- See Also:
- Constant Field Values
-
TOPOLOGY_UPDATE_REACTIVATE_WAIT_SECS
public static final String TOPOLOGY_UPDATE_REACTIVATE_WAIT_SECS
After updating a topology, amount of time to wait for it to come back up before reactivating it- See Also:
- Constant Field Values
-
TOPOLOGY_ENVIRONMENT
public static final String TOPOLOGY_ENVIRONMENT
Topology-specific environment properties to be added to an Heron instance. This is added to the existing environment (that of the Heron instance). This variable contains Map<String, String>- See Also:
- Constant Field Values
-
TOPOLOGY_TIMER_EVENTS
public static final String TOPOLOGY_TIMER_EVENTS
Timer events registered for a topology. This is a Map<String, Pair<Duration, Runnable>>. Where the key is the name and the value contains the frequency of the event and the task to run.- See Also:
- Constant Field Values
-
TOPOLOGY_REMOTE_DEBUGGING_ENABLE
public static final String TOPOLOGY_REMOTE_DEBUGGING_ENABLE
Enable Remote debugging for java heron instances- See Also:
- Constant Field Values
-
TOPOLOGY_DROPTUPLES_UPON_BACKPRESSURE
public static final String TOPOLOGY_DROPTUPLES_UPON_BACKPRESSURE
Do we want to drop tuples instead of initiating Spout BackPressure- See Also:
- Constant Field Values
-
TOPOLOGY_COMPONENT_OUTPUT_BPS
public static final String TOPOLOGY_COMPONENT_OUTPUT_BPS
The per component output bytes per second (rate limit) in this topology. It works with the addConfiguration() function in ComponentConfigurationDeclarer class. Example: builder.setSpout(...).addConfiguration(Config.TOPOLOGY_COMPONENT_OUTPUT_BPS, 1000); builder.setBolt(...).addConfiguration(Config.TOPOLOGY_COMPONENT_OUTPUT_BPS, 1000);- See Also:
- Constant Field Values
-
-
Method Detail
-
setTopologyCapTicket
public static void setTopologyCapTicket(Map<String,Object> conf, String ticket)
-
setTopologyProjectName
public static void setTopologyProjectName(Map<String,Object> conf, String project)
-
setSerializationClassName
public static void setSerializationClassName(Map<String,Object> conf, String className)
-
setKryoFactory
public static void setKryoFactory(Map conf, Class<? extends IKryoFactory> klass)
-
setSkipMissingKryoRegistrations
public static void setSkipMissingKryoRegistrations(Map conf, boolean skip)
-
registerKryoSerialization
public static void registerKryoSerialization(Map<String,Object> conf, Class klass)
-
registerKryoSerialization
public static void registerKryoSerialization(Map<String,Object> conf, Class klass, Class<? extends com.esotericsoftware.kryo.Serializer> serializerClass)
-
registerDecorator
public static void registerDecorator(Map conf, Class<? extends IKryoDecorator> klass)
-
setFallBackOnJavaSerialization
public static void setFallBackOnJavaSerialization(Map conf, boolean fallback)
-
setEnableAcking
@Deprecated public static void setEnableAcking(Map<String,Object> conf, boolean acking)
Deprecated.Is topology running with acking enabled?
-
setComponentParallelism
public static void setComponentParallelism(Map<String,Object> conf, int parallelism)
-
setTickTupleFrequency
public static void setTickTupleFrequency(Map<String,Object> conf, int seconds)
-
setTickTupleFrequencyMs
public static void setTickTupleFrequencyMs(Map<String,Object> conf, long millis)
-
setTopologyReliabilityMode
public static void setTopologyReliabilityMode(Map<String,Object> conf, Config.TopologyReliabilityMode mode)
-
setContainerCpuRequested
public static void setContainerCpuRequested(Map<String,Object> conf, double ncpus)
-
setContainerDiskRequested
@Deprecated public static void setContainerDiskRequested(Map<String,Object> conf, long nbytes)
Deprecated.use setContainerDiskRequested(Map<String, Object> conf, ByteAmount nbytes)Users should use the version of this method at uses ByteAmount
-
setContainerDiskRequested
public static void setContainerDiskRequested(Map<String,Object> conf, org.apache.heron.common.basics.ByteAmount nbytes)
-
setContainerRamRequested
@Deprecated public static void setContainerRamRequested(Map<String,Object> conf, long nbytes)
Deprecated.use setContainerRamRequested(Map<String, Object> conf, ByteAmount nbytes)Users should use the version of this method at uses ByteAmount
-
setContainerRamRequested
public static void setContainerRamRequested(Map<String,Object> conf, org.apache.heron.common.basics.ByteAmount nbytes)
-
setContainerMaxCpuHint
public static void setContainerMaxCpuHint(Map<String,Object> conf, double ncpus)
-
setContainerMaxDiskHint
public static void setContainerMaxDiskHint(Map<String,Object> conf, org.apache.heron.common.basics.ByteAmount nbytes)
-
setContainerMaxRamHint
public static void setContainerMaxRamHint(Map<String,Object> conf, org.apache.heron.common.basics.ByteAmount nbytes)
-
setContainerPaddingPercentage
public static void setContainerPaddingPercentage(Map<String,Object> conf, int percentage)
-
setContainerRamPadding
public static void setContainerRamPadding(Map<String,Object> conf, org.apache.heron.common.basics.ByteAmount nbytes)
-
setContainerCpuPadding
public static void setContainerCpuPadding(Map<String,Object> conf, double ncpus)
-
setMaxNumInstancesPerContainer
public static void setMaxNumInstancesPerContainer(Map<String,Object> conf, int n)
-
setComponentDiskMap
public static void setComponentDiskMap(Map<String,Object> conf, String diskMap)
-
setFFDSortingStrategy
public static void setFFDSortingStrategy(Map<String,Object> conf, String sortingStrategy)
-
setTopologyComponentOutputBPS
@Deprecated public static void setTopologyComponentOutputBPS(Map<String,Object> conf, long bps)
Deprecated.use the TOPOLOGY_COMPONENT_OUTPUT_BPS config with ComponentConfigurationDeclarer's addConfiguration() instead. Example: builder.setSpout(...).addConfiguration(Config.TOPOLOGY_COMPONENT_OUTPUT_BPS, 1000);This function should not be used to set rate limiter in topology config.
-
setComponentRam
@Deprecated public static void setComponentRam(Map<String,Object> conf, String component, long ramInBytes)
Deprecated.use setComponentRam(Map<String, Object> conf, String component, ByteAmount ramInBytes)Users should use the version of this method at uses ByteAmount
-
setComponentCpu
public static void setComponentCpu(Map<String,Object> conf, String component, double cpu)
-
setComponentRam
public static void setComponentRam(Map<String,Object> conf, String component, org.apache.heron.common.basics.ByteAmount ramInBytes)
-
setComponentDisk
public static void setComponentDisk(Map<String,Object> conf, String component, org.apache.heron.common.basics.ByteAmount diskInBytes)
-
setComponentJvmOptions
public static void setComponentJvmOptions(Map<String,Object> conf, String component, String jvmOptions)
-
setTopologyStatefulCheckpointIntervalSecs
public static void setTopologyStatefulCheckpointIntervalSecs(Map<String,Object> conf, int secs)
-
setTopologyStatefulStartClean
public static void setTopologyStatefulStartClean(Map<String,Object> conf, boolean clean)
-
setCheckpointManagerRam
public static void setCheckpointManagerRam(Map<String,Object> conf, org.apache.heron.common.basics.ByteAmount ramInBytes)
-
setTopologyStatefulSpillState
public static void setTopologyStatefulSpillState(Map<String,Object> conf, String spillState)
-
setTopologyStatefulSpillStateLocation
public static void setTopologyStatefulSpillStateLocation(Map<String,Object> conf, String location)
-
setStreamManagerRam
public static void setStreamManagerRam(Map<String,Object> conf, org.apache.heron.common.basics.ByteAmount ramInBytes)
-
setMetricsmgrRam
public static void setMetricsmgrRam(Map<String,Object> conf, org.apache.heron.common.basics.ByteAmount ramInBytes)
-
registerTopologyTimerEvents
public static void registerTopologyTimerEvents(Map<String,Object> conf, String name, Duration interval, Runnable task)
Registers a timer event that executes periodically- Parameters:
conf- the map with the existing topology configsname- the name of the timerinterval- the frequency in which to run the tasktask- the task to run
-
setDebug
public void setDebug(boolean isOn)
-
setTeamName
public void setTeamName(String teamName)
-
setTeamEmail
public void setTeamEmail(String teamEmail)
-
setTopologyCapTicket
public void setTopologyCapTicket(String ticket)
-
setTopologyProjectName
public void setTopologyProjectName(String project)
-
setNumStmgrs
public void setNumStmgrs(int stmgrs)
-
setSerializationClassName
public void setSerializationClassName(String className)
-
setKryoFactory
public void setKryoFactory(Class<? extends IKryoFactory> klass)
-
setSkipMissingKryoRegistrations
public void setSkipMissingKryoRegistrations(boolean skip)
-
registerKryoSerialization
public void registerKryoSerialization(Class klass)
-
registerKryoSerialization
public void registerKryoSerialization(Class klass, Class<? extends com.esotericsoftware.kryo.Serializer> serializerClass)
-
registerDecorator
public void registerDecorator(Class<? extends IKryoDecorator> klass)
-
setFallBackOnJavaSerialization
public void setFallBackOnJavaSerialization(boolean fallback)
-
setEnableAcking
@Deprecated public void setEnableAcking(boolean acking)
Deprecated.Is topology running with acking enabled? The SupressWarning will be removed once TOPOLOGY_ENABLE_ACKING is removed
-
setMessageTimeoutSecs
public void setMessageTimeoutSecs(int secs)
-
setTopologyReliabilityMode
public void setTopologyReliabilityMode(Config.TopologyReliabilityMode mode)
-
setComponentParallelism
public void setComponentParallelism(int parallelism)
-
setMaxSpoutPending
public void setMaxSpoutPending(int max)
-
setTickTupleFrequency
public void setTickTupleFrequency(int seconds)
-
setContainerCpuRequested
public void setContainerCpuRequested(double ncpus)
-
setContainerDiskRequested
public void setContainerDiskRequested(org.apache.heron.common.basics.ByteAmount nbytes)
-
setContainerRamRequested
public void setContainerRamRequested(org.apache.heron.common.basics.ByteAmount nbytes)
-
setContainerMaxCpuHint
public void setContainerMaxCpuHint(double ncpus)
-
setContainerMaxDiskHint
public void setContainerMaxDiskHint(org.apache.heron.common.basics.ByteAmount nbytes)
-
setContainerMaxRamHint
public void setContainerMaxRamHint(org.apache.heron.common.basics.ByteAmount nbytes)
-
setContainerPaddingPercentage
public void setContainerPaddingPercentage(int percentage)
-
setContainerRamPadding
public void setContainerRamPadding(org.apache.heron.common.basics.ByteAmount nbytes)
-
setContainerCpuPadding
public void setContainerCpuPadding(double ncpus)
-
setTopologyContainerMaxNumInstances
public void setTopologyContainerMaxNumInstances(int n)
-
setComponentCpuMap
public void setComponentCpuMap(String cpuMap)
-
setComponentRamMap
public void setComponentRamMap(String ramMap)
-
setComponentDiskMap
public void setComponentDiskMap(String diskMap)
-
setComponentCpu
public void setComponentCpu(String component, double cpu)
-
setComponentRam
public void setComponentRam(String component, org.apache.heron.common.basics.ByteAmount ramInBytes)
-
setComponentDisk
public void setComponentDisk(String component, org.apache.heron.common.basics.ByteAmount diskInBytes)
-
setFFDSortingStrategy
public void setFFDSortingStrategy(String sortingStrategy)
-
setUpdateDeactivateWaitDuration
public void setUpdateDeactivateWaitDuration(int seconds)
-
setUpdateReactivateWaitDuration
public void setUpdateReactivateWaitDuration(int seconds)
-
setTopologyStatefulCheckpointIntervalSecs
public void setTopologyStatefulCheckpointIntervalSecs(int secs)
-
setTopologyStatefulStartClean
public void setTopologyStatefulStartClean(boolean clean)
-
setCheckpointManagerRam
public void setCheckpointManagerRam(org.apache.heron.common.basics.ByteAmount ramInBytes)
-
setTopologyStatefulSpillState
public void setTopologyStatefulSpillState(String spillState)
-
setTopologyStatefulSpillStateLocation
public void setTopologyStatefulSpillStateLocation(String location)
-
setStreamManagerRam
public void setStreamManagerRam(org.apache.heron.common.basics.ByteAmount ramInBytes)
-
setMetricsmgrRam
public void setMetricsmgrRam(org.apache.heron.common.basics.ByteAmount ramInBytes)
-
registerTopologyTimerEvents
public void registerTopologyTimerEvents(String name, Duration interval, Runnable task)
-
setTopologyRemoteDebugging
public void setTopologyRemoteDebugging(boolean isOn)
-
setTopologyDropTuplesUponBackpressure
public void setTopologyDropTuplesUponBackpressure(boolean dropTuples)
-
setTopologyComponentOutputBPS
@Deprecated public void setTopologyComponentOutputBPS(long bps)
Deprecated.use the TOPOLOGY_COMPONENT_OUTPUT_BPS config with ComponentConfigurationDeclarer's addConfiguration() instead. Example: builder.setSpout(...).addConfiguration(Config.TOPOLOGY_COMPONENT_OUTPUT_BPS, 1000);This function should not be used to set rate limiter in topology config.
-
-