Metadata-Version: 2.1
Name: dnnnlp
Version: 1.1.4
Summary: Deep Neural Networks for Natural Language Processing classification or sequential task written by PyTorch.
Home-page: https://github.com/NUSTM/pytorch-dnnnlp
Author: KzXuan
Author-email: kaizhouxuan@gmail.com
License: MIT
Description: ## PyTorch - Deep Neural Network - Natural Language Processing
        
        Version 1.1 by KzXuan
        
        **Contains CNN, RNN and Transformer layers and models implemented by pytorch for classification and sequence labeling tasks in NLP.**
        
        * Newly designed modules.
        * Reduce usage complexity.
        * Use `mask` as the sequence length identifier.
        * Multi-GPU parallel for grid search.
        
        <br>
        
        ### Dependecies
        
        python 3.5+ & pytorch 1.2.0+
        
        <br>
        
        ### [Install](https://pypi.org/project/dnnnlp/)
        
        ```bash
        > pip install dnnnlp
        ```
        
        <br>
        
        ### [API Document](./docs.md) (In Chinese)
        
          * [Layer](./docs.md#Layer) - [layer.py](./dnnnlp/layer.py)
          * [Model](./docs.md#Model) -  [model.py](./dnnnlp/model.py)
          * [Execution](./docs.md#Execution) - [exec.py](./dnnnlp/exec.py)
          * [Utility](./docs.md#Utility) - [utils.py](./dnnnlp/utils.py)
        
        <br>
        
        ### Hyperparameters
        
        | Name          | Type  | Default     | Description                                                    |
        | ------------- | ----- | ----------- | -------------------------------------------------------------- |
        | n_gpu         | int   | 1           | The number of GPUs (0 means no GPU acceleration).              |
        | space_turbo   | bool  | True        | Accelerate with more GPU memories.                             |
        | rand_seed     | int   | 100         | Random seed setting.                                           |
        | data_shuffle  | bool  | Ture        | Disrupt data for training.                                     |
        | emb_type      | str   | None        | Embedding modes contain None, 'const' or 'variable'.           |
        | emb_dim       | int   | 300         | Embedding dimension (or feature dimension).                    |
        | n_class       | int   | 2           | Number of target classes.                                      |
        | n_hidden      | int   | 50          | Number of hidden nodes, or output channels of CNN.             |
        | learning_rate | float | 0.01        | Learning rate.                                                 |
        | l2_reg        | float | 1e-6        | L2 regular.                                                    |
        | batch_size    | int   | 32          | Number of samples for one batch.                               |
        | iter_times    | int   | 30          | Number of iterations.                                          |
        | display_step  | int   | 2           | The number of iterations between each output of the result.    |
        | drop_prob     | float | 0.1         | Dropout ratio.                                                 |
        | eval_metric   | str   | 'accuracy'  | Evaluation metrics contain 'accuracy', 'macro', 'class1', etc. |
        
        <br>
        
        ### Usage
        
        ```python
        # import our modules
        from dnnnlp.model import RNNModel
        from dnnnlp.exec import default_args, Classify
        
        # load the embedding matrix
        emb_mat = np.array((-1, 300))
        # load the train data
        train_x = np.array((800, 50))
        train_y = np.array((800,))
        train_mask = np.array((800, 50))
        # load the test data
        test_x = np.array((200, 50))
        test_y = np.array((200,))
        test_mask = np.array((200, 50))
        
        # get the default arguments
        args = default_args()
        # modify part of the arguments
        args.space_turbo = False
        args.n_hidden = 100
        args.batch_size = 32
        ```
        
        * Classification
        
        ```python
        # initilize a model
        model = RNNModel(args, emb_mat, bi_direction=False, rnn_type='GRU', use_attention=True)
        # initilize a classifier
        nn = Classify(model, args, train_x, train_y, train_mask, test_x, test_y, test_mask)
        # do training and testing
        evals = nn.train_test(device_id=0)
        ```
        
        * Run several times and get the average score.
        
        ````python
        # initilize a model
        model = CNNModel(args, emb_mat, kernel_widths=[2, 3, 4])
        # initilize a classifier
        nn = Classify(model, args, train_x, train_y, train_mask)
        # run the model several times
        avg_evals = average_several_run(nn.train_test, args, n_times=8, n_paral=4, fold=5)
        ````
        
        * Parameters' grid search.
        
        ````python
        # initilize a model
        model = TransformerModel(args, n_layer=12, n_head=8)
        # initilize a classifier
        nn = Classify(model, args, train_x, train_y, train_mask, test_x, test_y, test_mask)
        # set searching params
        params_search = {'learning_rate': [0.1, 0.01], 'n_hidden': [50, 100]}
        # run grid search
        max_evals = grid_search(nn, nn.train_test, args, params_search)
        ````
        
        * Sequence labeling
        
        ```python
        from dnnnlp.model import RNNCRFModel
        from dnnnlp.exec import default_args, SequenceLabeling
        
        # load the train data
        train_x = np.array((1000, 50))
        train_y = np.array((1000, 50))
        train_mask = np.array((1000, 50))
        
        # initilize a model
        model = RNNCRFModel(args)
        # initilize a labeler
        nn = SequenceLabeling(model, args, train_x, train_y, train_mask)
        # do cross validation
        nn.cross_validation(fold=5)
        ```
        
        <br>
        
        ### History
        
        **version 1.1**
          * Add `CRFLayer`: packaging CRF for both training and testing.
          * Add `RNNCRFModel`: a integrated RNN-CRF sequence labeling model.
          * Add `SequenceLabeling`: a sequence labeling execution module that inherits from Classify.
          * Fix errors in judging whether a tensor is None.
        
        **version 1.0**
          * Rename project `dnn` to `dnnnlp`.
          * Remove file `base`, add file `utils`.
          * Optimize and rename `SoftmaxLayer` and `SoftAttentionLayer`.
          * Rewrite and rename `EmbeddingLayer`, `CNNLayer` and `RNNLayer`.
          * Rewrite `MultiheadAttentionLayer`: a packaging attention layer based on `nn.MultiheadAttention`.
          * Rewrite `TransformerLayer`: support new `MultiheadAttentionLayer`.
          * Optimize and rename `CNNModel`, `RNNModel` and `TransformerModel`.
          * Optimize and rename `Classify`: a highly applicable classification execution module.
          * Rewrite `average_several_run` and `grid_search`: support multi-GPU parallel.
          * Support pytorch 1.2.0.
        
        **version 0.12**
          * Update `RNN_layer`: fully support for tanh, LSTM and GRU.
          * Fix errors in some mask operations.
          * Support pytorch 1.1.0.
        
        Old version [0.12.3](https://github.com/NUSTM/pytorch-dnnnlp/tree/8d2d6c4e432076e13020ae54954aa419f3bb9bce).
        
        **version 0.11**
          * Provides an acceleration method by using more GPU memories.
          * Fix the problem of memory consumption caused by abnormal data reading.
          * Add `multi_head_attention_layer`: packaging multi-head attention for Transformer.
          * Add `Transformer_layer` and `Transformer_model`: packaging Transformer layer and model written by ourself.
          * Support data disruption for training.
        
        **version 0.10**
          * Split the code into four files: `base`, `layer`, `model`, `exec`.
          * Add `CNN_layer` and `CNN_model`: packaging CNN layer and model.
          * Support multi-GPU parallel for each model.
        
        **version 0.9**
          * Fix the problem of output format.
          * Fix the statistical errors in cross-validation part of `LSTM_classify`.
          * Rename: `LSTM_model` to `RNN_layer`, `self_attention` to `self_attention_layer`.
          * Add `softmax_layer`: a packaging fully-connected layer.
        
        **version 0.8**
          * Adjust the applicability of functions in `LSTM_classify` to avoid rewriting in `LSTM_sequence`.
          * Optimize the way of parameter transfer.
          * A more complete evaluation mechanism.
        
        **version 0.7**
          * Add `LSTM_sequence`: a sequence labeling module for `LSTM_model`.
          * Fix the nan-value problem in hierarchical classification.
          * Support pytorch 1.0.0.
        
        **version 0.6**
          * Update `LSTM_classify`: support hierarchical classification.
          * The `GRU_model` is merged into the `LSTM_model`.
          * Adapt to CPU operation.
        
        **version 0.5**
          * Split the running part of `LSTM_classify` to reduce the rewrite of custom models.
          * Add control for visual output.
          * Create function `average_several_run`: support to get the average score after several training and testing.
          * Create function `grid_search`: support parameters' grid search.
        
        **version 0.4**
          * Add `GRU_model`: a packaging GRU model based on `nn.GRU`.
          * Support L2 regular.
        
        **version 0.3**
          * Add `self_attention`: provides attention mechanism support.
          * Update `LSTM_classify`: adapts to complex custom models.
        
        **version 0.2**
          * Support mode selection of embedding.
          * Default usage of `nn.Dropout`.
          * Create function `default_args` to provide default hyperparameters.
        
        **version 0.1**
          * Initilization of project `dnn`: based on pytorch 0.4.1.
          * Add `LSTM_model`: a packaging LSTM model based on `nn.LSTM`.
          * Add `LSTM_classify`: a classification module for LSTM model, which supports train-test and corss-validation.
        
Platform: UNKNOWN
Classifier: License :: Free for non-commercial use
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Programming Language :: Python :: 3.5
Classifier: Programming Language :: Python :: 3.5
Classifier: Programming Language :: Python :: 3.6
Classifier: Programming Language :: Python :: 3.7
Description-Content-Type: text/markdown
