Package opennlp.tools.tokenize

## Class DictionaryDetokenizer

- java.lang.Object
- - opennlp.tools.tokenize.DictionaryDetokenizer

- All Implemented Interfaces:
    `Detokenizer`

      public class DictionaryDetokenizer
      extends Object
      implements Detokenizer

  A rule based detokenizer. Simple rules which indicate in which direction a token should be moved are looked up in a `dictionary`.
  See Also:
    `Detokenizer`, `DetokenizationDictionary`

- - ### Nested Class Summary

    - ### Nested classes/interfaces inherited from interface opennlp.tools.tokenize.Detokenizer

      `Detokenizer.DetokenizationOperation`

  - ### Constructor Summary

    Constructors
    Constructor | Description
    --- | ---
    `DictionaryDetokenizer​(DetokenizationDictionary dict)` | Initializes a `DictionaryDetokenizer` instance.

  - ### Method Summary

    All Methods Instance Methods Concrete Methods
    Modifier and Type | Method | Description
    --- | --- | ---
    `Detokenizer.DetokenizationOperation[]` | `detokenize​(String[] tokens)` | Detokenizes the collection of tokens.
    `String` | `detokenize​(String[] tokens, String splitMarker)` | Detokenizes the input `tokens` into a String.

    - ### Methods inherited from class java.lang.Object

      `equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- - ### Constructor Detail

    - #### DictionaryDetokenizer

          public DictionaryDetokenizer​(DetokenizationDictionary dict)

      Initializes a `DictionaryDetokenizer` instance.
      Parameters:
        `dict` - The `DetokenizationDictionary` to be used.

  - ### Method Detail

    - #### detokenize

          public Detokenizer.DetokenizationOperation[] detokenize​(String[] tokens)

      Description copied from interface: `Detokenizer`
      Detokenizes the collection of tokens.
      Specified by:
        `detokenize` in interface `Detokenizer`
      Parameters:
        `tokens` - The elements which should be detokenized.
      Returns:
        The `merge operations` to handle given `tokens`.

    - #### detokenize

          public String detokenize​(String[] tokens,
                                   String splitMarker)

      Description copied from interface: `Detokenizer`
      Detokenizes the input `tokens` into a String. Tokens which are connected without a `whitespace` character in between can be separated by a given `splitMarker`.
      Specified by:
        `detokenize` in interface `Detokenizer`
      Parameters:
        `tokens` - The elements which should be concatenated.
        `splitMarker` - The split marker or `null`.
      Returns:
        The concatenated tokens as a single string.