NLP / Language Models
720 views

Bloom Model for Bangla

PyTorchBLOOMFine-TuningLoRAQuantizationTransformersWandB
Completed

A language model based on BLOOM, fine-tuned specifically for Bangla language understanding and generation, with support for bilingual Bangla-English instruction following.

Problem

Limited availability of high-quality language models for Bengali language speakers.

Solution

Fine-tuned the BLOOM 3B model on a curated dataset of conversational Bangla, using LoRA for efficient adaptation and maintaining bilingual capabilities.