Some utililties to help you using fastai2 to train huggingface transformers.

Install

pip3 install git+https://github.com/cwza/fastai_transformers_utils.git

How to use

Please see the notebooks that named start with example and under nbs directory.

How to develop

Editable install

git clone https://github.com/cwza/fastai_transformers_utils
cd fastai_transformers_utils
make install

Develop pipeline

  1. Modify notebooks in nbs folder (Write unit tests in the same notebook and create new notebook to write integration test)
  2. make build-lib to update python files
  3. make test to run unit test
  4. make test-slow to run integration test
  5. make build-all to run build-lib, build-docs, clean-nbs
  6. git add commit and push