![Vintern-1B ❄️ (Viet-InternVL2-1B) [🤗 HF Demo] - The LLaV...](/static/https://cdn5.telesco.pe/file/LWVvNJ9q_daVLwCoMYIW1Xls3dUc3ONte1ApsXBzQH3DwSRmINA8DsWDrGliJ7nNRoRPJIPM9cL83XYITa2TTh0gM-64rxwxP07k20SaS_JCCJ4UfTlK_vzoOhMjDHxYp_pqThHkJl_qCFCNL84_igjQ4tdglBZ_yEvF4aiZvYJ6ODsrV0b94n4atN3ZxeT8FWELgdpT4m3xjG7Uu8_FitoDixWO6JD__V3wYNf4V3hGVKDg4UZA4v0NnkdaPaPW6NqTnhO0cc2DH8jNpxXdIhOGeev9imG8b9JrAxM9NPXzAGgOGsoVwQGFWamIahSIaZEzCqdd8oPKc5vnCy5a4g.jpg)
Vintern-1B ❄️ (Viet-InternVL2-1B) [🤗 HF Demo] - The LLaVA 🌋 Challenger
We are excited to introduce Vintern-1B the Vietnamese 🇻🇳 multimodal model that combines the advanced Vietnamese language model Qwen2-0.5B-Instruct with the latest visual model, InternViT-300M-448px, CVPR 2024. This model excels in tasks such as OCR-VQA, Doc-VQA, and Chart-VQA,... With only 1 billion parameters, it is 4096 context length finetuned from the InternVL2-1B model on over 1.5 million specialized image-question-answer pairs for optical character recognition 🔍, text recognition 🔤, document extraction 📑, and more. The model can be integrated into various on-device applications 📱, demonstrating its versatility and robust capabilities.
Vintern-1B is a multimodal large language model series, featuring models of various sizes. For each size, we release instruction-tuned models optimized for multimodal tasks. Vintern-1B consists of InternViT-300M-448px, an MLP projector, and Qwen2-0.5B-Instruct.
Training details 📚
The fine-tuning dataset was meticulously sampled in part from the following datasets:
Viet-OCR-VQA
Viet-Doc-VQA
Viet-Doc-VQA-II
Benchmarks 📈
Since there are still many different metrics that need to be tested, we chose a quick and simple metric first to guide the development of our model. Our metric is inspired by Lavy's paper. For the time being, we are using GPT-4 to evaluate the quality of answers on two datasets: OpenViVQA and ViTextVQA. Detailed results can be found at the provided Here. The inputs are images, questions, labels, and predicted answers. The model will return a score from 0 to 10 for the corresponding answer quality. The results table is shown below.https://huggingface.co/5CD-AI/Viet-InternVL2-1B
We are excited to introduce Vintern-1B the Vietnamese 🇻🇳 multimodal model that combines the advanced Vietnamese language model Qwen2-0.5B-Instruct with the latest visual model, InternViT-300M-448px, CVPR 2024. This model excels in tasks such as OCR-VQA, Doc-VQA, and Chart-VQA,... With only 1 billion parameters, it is 4096 context length finetuned from the InternVL2-1B model on over 1.5 million specialized image-question-answer pairs for optical character recognition 🔍, text recognition 🔤, document extraction 📑, and more. The model can be integrated into various on-device applications 📱, demonstrating its versatility and robust capabilities.
Vintern-1B is a multimodal large language model series, featuring models of various sizes. For each size, we release instruction-tuned models optimized for multimodal tasks. Vintern-1B consists of InternViT-300M-448px, an MLP projector, and Qwen2-0.5B-Instruct.
Training details 📚
The fine-tuning dataset was meticulously sampled in part from the following datasets:
Viet-OCR-VQA
Viet-Doc-VQA
Viet-Doc-VQA-II
Benchmarks 📈
Since there are still many different metrics that need to be tested, we chose a quick and simple metric first to guide the development of our model. Our metric is inspired by Lavy's paper. For the time being, we are using GPT-4 to evaluate the quality of answers on two datasets: OpenViVQA and ViTextVQA. Detailed results can be found at the provided Here. The inputs are images, questions, labels, and predicted answers. The model will return a score from 0 to 10 for the corresponding answer quality. The results table is shown below.https://huggingface.co/5CD-AI/Viet-InternVL2-1B
huggingface.co
5CD-AI/Viet-InternVL2-1B · Hugging Face
We’re on a journey to advance and democratize artificial intelligence through open source and open science.