## [0.0.26](https://github.com/mmmwhy/pure_attention/compare/v0.0.25...v0.0.26) (2022-02-13)


### Bug Fixes

* **type:** fix some code mistake ([3e1a81d](https://github.com/mmmwhy/pure_attention/commit/3e1a81dd351f2a31ca03fce7cf8ca80be2b94a6d))


### Features

* **ddp:** logger when rank == 0, or no ddp ([792709b](https://github.com/mmmwhy/pure_attention/commit/792709b2547ee9c5644638f8b8e701ec1fb3f6fd))
* **decode:** add some transformer decode code ([52b044b](https://github.com/mmmwhy/pure_attention/commit/52b044b0fa79dcb3b9ba8fcd2747f05bc43de808))
* **schedule:** cosine schedule with warmup ([87c8886](https://github.com/mmmwhy/pure_attention/commit/87c88865a685bcf5bb2806d15ae7a1a243676509))


### Performance Improvements

* **runner:** logger missing_keys and unexpected_key in runner ([a9e3ff9](https://github.com/mmmwhy/pure_attention/commit/a9e3ff9ca7771fac648c427a98d0dd5414956cbd))



## [0.0.20](https://github.com/mmmwhy/pure_attention/compare/eb61b313458ac18bf4b15271fee2cf7e39f8afde...v0.0.20) (2022-02-02)


### Bug Fixes

* **bert:** change LayerNorm to layer_norm ([a99831e](https://github.com/mmmwhy/pure_attention/commit/a99831ee3b4ad06cadbb0262720c0836717d7508))


### Features

* **bert:** add tokenizer part ([054df14](https://github.com/mmmwhy/pure_attention/commit/054df14c7dfefc0b2edb47824578b33f4a5c8539))
* **layers:** fix import for layerNorm ([eb61b31](https://github.com/mmmwhy/pure_attention/commit/eb61b313458ac18bf4b15271fee2cf7e39f8afde))
* **nlp:** init basic bert code ([f9cb13a](https://github.com/mmmwhy/pure_attention/commit/f9cb13a3e811eb8c44ba8ff1373d688311426927))



