How I see a possible integration with HuggingFace. We reuse best practices for training NNs in general, which are also implemented in Catalyst: - batch accumulation - warmup - nvidia apex support - Cycling Learning Rate - shedulers We also built upon Catalyst training environment: - configs - logging - monitoring training (w&b support) - reproducibility We can extend it with NLP-specific stuff: - custom loaders for different tasks - sequence bucketing - tensor trimming - ... Let's extend and elaborate.
How I see a possible integration with HuggingFace.
We reuse best practices for training NNs in general, which are also implemented in Catalyst:
We also built upon Catalyst training environment:
We can extend it with NLP-specific stuff:
Let's extend and elaborate.