Share and discover more about AI with social posts from the community.huggingface/OpenAi
๐ค ๐ง๐ต๐ฒ ๐๐ ๐ฆ๐ฐ๐ถ๐ฒ๐ป๐๐ถ๐๐: ๐๐ด๐ฒ๐ป๐๐ถ๐ฐ, ๐ณ๐๐น๐น๐-๐ฎ๐๐๐ผ๐บ๐ฎ๐๐ฒ๐ฑ ๐ฟ๐ฒ๐๐ฒ๐ฎ๐ฟ๐ฐ๐ต ๐ฝ๐ถ๐ฝ๐ฒ๐น๐ถ๐ป๐ฒ ๐ณ๐ผ๐ฟ ๐๐ป๐ฑ๐ฒ๐ฟ $๐ญ๐ฑ ๐ฝ๐ฒ๐ฟ ๐ฝ๐ฎ๐ฝ๐ฒ๐ฟ
Researchers have just created an AI system that ๐ฐ๐ฎ๐ป ๐ฐ๐ผ๐ป๐ฑ๐๐ฐ๐ ๐ฒ๐ป๐๐ถ๐ฟ๐ฒ ๐ฟ๐ฒ๐๐ฒ๐ฎ๐ฟ๐ฐ๐ต ๐ฝ๐ฟ๐ผ๐ท๐ฒ๐ฐ๐๐ ๐ณ๐ฟ๐ผ๐บ ๐๐๐ฎ๐ฟ๐ ๐๐ผ ๐ณ๐ถ๐ป๐ถ๐๐ต, ๐ฝ๐ผ๐๐ฒ๐ป๐๐ถ๐ฎ๐น๐น๐ ๐ฟ๐ฒ๐๐ผ๐น๐๐๐ถ๐ผ๐ป๐ถ๐๐ถ๐ป๐ด ๐ต๐ผ๐ ๐๐ฐ๐ถ๐ฒ๐ป๐๐ถ๐ณ๐ถ๐ฐ ๐ฑ๐ถ๐๐ฐ๐ผ๐๐ฒ๐ฟ๐ถ๐ฒ๐ ๐ฎ๐ฟ๐ฒ ๐บ๐ฎ๐ฑ๐ฒ.
It doesn't just assist with specific tasks - it automates the entire research process, from generating ideas to writing and reviewing papers.
1 - brainstorm novel research directions, 2- write and execute code for experiments & visualize results, get references, and even 3- write up findings in a full academic paper format!
And it can do all this for under $15 per paper! ๐คฏ
๐๐ฒ๐ ๐ถ๐ป๐๐ถ๐ด๐ต๐๐:
๐ง Generates novel research ideas across multiple topics (e.g. diffusion modeling, transformers, learning dynamics aka โgrokkingโ)
๐จโ๐ป Uses open-source coding assistant Aider to implement ideas and run experiments. This is especially important since this agentic assistant can iterate if it fails somewhere.
๐ Visualizes results and plans follow-up experiments (up to 5 rounds)
โ๏ธ Writes full academic papers, including finding references using Semantic Search API
๐ต๏ธ Runs a simulated peer review process to evaluate paper quality
๐ฐ Total cost per paper is under $15. This system can generate "hundreds of interesting, medium-quality papers" in just a week !
๐ฆ๐๐ถ๐น๐น ๐ป๐ผ๐ ๐ฟ๐ฒ๐ฎ๐ฑ๐ ๐๐ผ ๐ณ๐ถ๐น๐น ๐๐๐๐ฅ ๐๐ถ๐๐ต ๐ฝ๐ฎ๐ฝ๐ฒ๐ฟ๐:
๐ Ideas generated in one domain tend to be repetitive across different runs, and even different language model
๐ Does not use vision capabilities to fix visual issues in plots
๐ญ Models occasionally hallucinate entire results tables
โ Only few of the generated papers would actually meet the threshold for acceptance at a top AI conference
๐ Read their paper:
The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery (2408.06292) https://huggingface.co/papers/2408.06292
Researchers have just created an AI system that ๐ฐ๐ฎ๐ป ๐ฐ๐ผ๐ป๐ฑ๐๐ฐ๐ ๐ฒ๐ป๐๐ถ๐ฟ๐ฒ ๐ฟ๐ฒ๐๐ฒ๐ฎ๐ฟ๐ฐ๐ต ๐ฝ๐ฟ๐ผ๐ท๐ฒ๐ฐ๐๐ ๐ณ๐ฟ๐ผ๐บ ๐๐๐ฎ๐ฟ๐ ๐๐ผ ๐ณ๐ถ๐ป๐ถ๐๐ต, ๐ฝ๐ผ๐๐ฒ๐ป๐๐ถ๐ฎ๐น๐น๐ ๐ฟ๐ฒ๐๐ผ๐น๐๐๐ถ๐ผ๐ป๐ถ๐๐ถ๐ป๐ด ๐ต๐ผ๐ ๐๐ฐ๐ถ๐ฒ๐ป๐๐ถ๐ณ๐ถ๐ฐ ๐ฑ๐ถ๐๐ฐ๐ผ๐๐ฒ๐ฟ๐ถ๐ฒ๐ ๐ฎ๐ฟ๐ฒ ๐บ๐ฎ๐ฑ๐ฒ.
It doesn't just assist with specific tasks - it automates the entire research process, from generating ideas to writing and reviewing papers.
1 - brainstorm novel research directions, 2- write and execute code for experiments & visualize results, get references, and even 3- write up findings in a full academic paper format!
And it can do all this for under $15 per paper! ๐คฏ
๐๐ฒ๐ ๐ถ๐ป๐๐ถ๐ด๐ต๐๐:
๐ง Generates novel research ideas across multiple topics (e.g. diffusion modeling, transformers, learning dynamics aka โgrokkingโ)
๐จโ๐ป Uses open-source coding assistant Aider to implement ideas and run experiments. This is especially important since this agentic assistant can iterate if it fails somewhere.
๐ Visualizes results and plans follow-up experiments (up to 5 rounds)
โ๏ธ Writes full academic papers, including finding references using Semantic Search API
๐ต๏ธ Runs a simulated peer review process to evaluate paper quality
๐ฐ Total cost per paper is under $15. This system can generate "hundreds of interesting, medium-quality papers" in just a week !
๐ฆ๐๐ถ๐น๐น ๐ป๐ผ๐ ๐ฟ๐ฒ๐ฎ๐ฑ๐ ๐๐ผ ๐ณ๐ถ๐น๐น ๐๐๐๐ฅ ๐๐ถ๐๐ต ๐ฝ๐ฎ๐ฝ๐ฒ๐ฟ๐:
๐ Ideas generated in one domain tend to be repetitive across different runs, and even different language model
๐ Does not use vision capabilities to fix visual issues in plots
๐ญ Models occasionally hallucinate entire results tables
โ Only few of the generated papers would actually meet the threshold for acceptance at a top AI conference
๐ Read their paper:
The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery (2408.06292) https://huggingface.co/papers/2408.06292
Hey everyone ๐ค!
Check out this awesome new model for object segmentation!
finegrain/finegrain-object-cutter
.
We (finegrain) have trained this new model in partnership with Nfinite and some of their synthetic data, the resulting model is incredibly accurate ๐.
Itโs all open source under the MIT license (
finegrain/finegrain-box-segmenter
), complete with a test set tailored for e-commerce (
finegrain/finegrain-product-masks-lite
). Have fun experimenting with it!
Check out this awesome new model for object segmentation!
finegrain/finegrain-object-cutter
.
We (finegrain) have trained this new model in partnership with Nfinite and some of their synthetic data, the resulting model is incredibly accurate ๐.
Itโs all open source under the MIT license (
finegrain/finegrain-box-segmenter
), complete with a test set tailored for e-commerce (
finegrain/finegrain-product-masks-lite
). Have fun experimenting with it!
๐๐ปโโ๏ธhey there folks ,
โ๏ธInkubaLM has been trained from scratch using 1.9 billion tokens of data for five African languages, along with English and French data, totaling 2.4 billion tokens of data. It is capable of understanding and generating content in five African languages: Swahili, Yoruba, Hausa, isiZulu, and isiXhosa, as well as English and French.
model
lelapa/InkubaLM-0.4B
demo
Tonic/Inkuba-0.4B
โ๏ธInkubaLM has been trained from scratch using 1.9 billion tokens of data for five African languages, along with English and French data, totaling 2.4 billion tokens of data. It is capable of understanding and generating content in five African languages: Swahili, Yoruba, Hausa, isiZulu, and isiXhosa, as well as English and French.
model
lelapa/InkubaLM-0.4B
demo
Tonic/Inkuba-0.4B
Spent a few minutes to build an alternative to Character AI on top of llama3.1 405B through SambaNova's super fast inference API
Space:
kz919/Persona-AI
API referral link: https://sambanova.ai/fast-api?api_ref=907266
Space:
kz919/Persona-AI
API referral link: https://sambanova.ai/fast-api?api_ref=907266
I started training a public LoRA style (2 seperate training each on 4x A6000).
Experimenting captions vs non-captions. So we will see which yields best results for style training on FLUX.
Generated captions with multi-GPU batch Joycaption app.
I am showing 5 examples of what Joycaption generates on FLUX dev. Left images are the original style images from the dataset.
I used my multi-GPU Joycaption APP (used 8x A6000 for ultra fast captioning) : https://www.patreon.com/posts/110613301
I used my Gradio batch caption editor to edit some words and add activation token as ohwx 3d render : https://www.patreon.com/posts/108992085
The no caption dataset uses only ohwx 3d render as caption
I am using my newest 4x_GPU_Rank_1_SLOW_Better_Quality.json on 4X A6000 GPU and train 500 epochs โ 114 images : https://www.patreon.com/posts/110879657
Total step count is being 500 * 114 / 4 (4x GPU โ batch size 1) = 14250
Taking 37 hours currently if I donโt terminate early
Will save a checkpoint once every 25 epochs
Full Windows Kohya LoRA training tutorial : https://youtu.be/nySGu12Y05k
Full cloud tutorial I am still editing
Hopefully will share trained LoRA on Hugging Face and CivitAI along with full dataset including captions.
I got permission to share dataset but canโt be used commercially.
Also I will hopefully share full workflow in the CivitAI and Hugging Face LoRA pages.
Experimenting captions vs non-captions. So we will see which yields best results for style training on FLUX.
Generated captions with multi-GPU batch Joycaption app.
I am showing 5 examples of what Joycaption generates on FLUX dev. Left images are the original style images from the dataset.
I used my multi-GPU Joycaption APP (used 8x A6000 for ultra fast captioning) : https://www.patreon.com/posts/110613301
I used my Gradio batch caption editor to edit some words and add activation token as ohwx 3d render : https://www.patreon.com/posts/108992085
The no caption dataset uses only ohwx 3d render as caption
I am using my newest 4x_GPU_Rank_1_SLOW_Better_Quality.json on 4X A6000 GPU and train 500 epochs โ 114 images : https://www.patreon.com/posts/110879657
Total step count is being 500 * 114 / 4 (4x GPU โ batch size 1) = 14250
Taking 37 hours currently if I donโt terminate early
Will save a checkpoint once every 25 epochs
Full Windows Kohya LoRA training tutorial : https://youtu.be/nySGu12Y05k
Full cloud tutorial I am still editing
Hopefully will share trained LoRA on Hugging Face and CivitAI along with full dataset including captions.
I got permission to share dataset but canโt be used commercially.
Also I will hopefully share full workflow in the CivitAI and Hugging Face LoRA pages.
# Excited to Share: New LLM Tokenization - Convert Text to tokens and vice versa! ๐
I've just developed a powerful tool for anyone working with Language Models (LLMs) or diving into Natural Language Processing (NLP).
๐ Introducing the LLM Tokenization - Convert Text to tokens and vice versa!!
Key Features:
- Convert text to tokens and token IDs
- Reverse engineer: convert token IDs back to text
- Support for popular models: LLama3 (Will add more models iteratively)
- User-friendly Gradio interface for easy interaction
Whether you're debugging your NLP pipeline, exploring how different models tokenize text, or just curious about the inner workings of LLMs, this tool is for you!
๐ฉโ๐ป Tech Stack:
- Python
- Gradio for the web interface
- Hugging Face Transformers for tokenization
The application is deployed in Hugging Face spaces as Gradio application
๐ Try it out: https://lnkd.in/g6R5z9k2
#NLP #MachineLearning #AI #PythonDevelopment #OpenSourceAI
I've just developed a powerful tool for anyone working with Language Models (LLMs) or diving into Natural Language Processing (NLP).
๐ Introducing the LLM Tokenization - Convert Text to tokens and vice versa!!
Key Features:
- Convert text to tokens and token IDs
- Reverse engineer: convert token IDs back to text
- Support for popular models: LLama3 (Will add more models iteratively)
- User-friendly Gradio interface for easy interaction
Whether you're debugging your NLP pipeline, exploring how different models tokenize text, or just curious about the inner workings of LLMs, this tool is for you!
๐ฉโ๐ป Tech Stack:
- Python
- Gradio for the web interface
- Hugging Face Transformers for tokenization
The application is deployed in Hugging Face spaces as Gradio application
๐ Try it out: https://lnkd.in/g6R5z9k2
#NLP #MachineLearning #AI #PythonDevelopment #OpenSourceAI
๐๐๐ฐ ๐๐๐ฅ๐๐๐ฌ๐: ๐๐๐ฃ๐จ๐ซ ๐๐๐ ๐๐ข๐ ๐ข๐ญ๐๐ฅ ๐๐ฅ๐๐ฏ๐๐ญ๐ข๐จ๐ง ๐๐จ๐๐๐ฅ ๐๐ฑ๐ฉ๐๐ง๐ฌ๐ข๐จ๐ง ๐บ
Dataset:
Major-TOM/Core-DEM
Today with European Space Agency - ESA and Adobe Research, we release a global expansion to Major TOM with GLO-30 DEM data.
You can now instantly access nearly 2M of Major TOM samples with elevation data to build your next AI model for EO. ๐
๐ Browse the data in our usual viewer app:
Major-TOM/MajorTOM-Core-Viewer
Fantastic work championed by Paul Borne--Pons @NewtNewt ๐
Dataset:
Major-TOM/Core-DEM
Today with European Space Agency - ESA and Adobe Research, we release a global expansion to Major TOM with GLO-30 DEM data.
You can now instantly access nearly 2M of Major TOM samples with elevation data to build your next AI model for EO. ๐
๐ Browse the data in our usual viewer app:
Major-TOM/MajorTOM-Core-Viewer
Fantastic work championed by Paul Borne--Pons @NewtNewt ๐
๐๐ฒ ๐๐ข๐ซ๐ฌ๐ญ ๐๐จ๐ฆ๐ฆ๐ฎ๐ง๐ข๐ญ๐ฒ ๐๐ซ๐ญ๐ข๐๐ฅ๐! ๐๐๐ฅ๐๐๐ญ๐ข๐ฏ๐ ๐๐ข๐ง๐-๐ญ๐ฎ๐ง๐ข๐ง๐ ๐ฐ๐ข๐ญ๐ก ๐๐ฉ๐๐๐ญ๐ซ๐ฎ๐ฆ ๐ฏ
Full walkthrough on how to get started with Spectrum and TRL for efficient fine-tuning.
๐ ๐ฃ https://huggingface.co/blog/anakin87/spectrum
---
Looking to fine-tune Language Models efficiently and save on computational resources?
One popular method is QLoRa, which quantizes the original model and trains low-rank adapters on top.
It's quite effective and uses less GPU than full fine-tuning.
However, QLoRa applies Low-Rank Adaptation uniformly across the entire model.
What if we could identify the most informative layers and only fine-tune those? ๐ค
This is exactly what Spectrum does! ๐
๐ฌ Spectrum analyzes the weight matrices for all layers in a Language Model and calculates a Signal to Noise Ratio (SNR) for each one.
(It uses Random Matrix Theory and Marchenko-Pastur distribution to distinguish signal from noise.)
๐ฏ Based on a chosen percentage (say, 25%), Spectrum selects the most informative layers of each type (mlp.down_proj, self_attn.o_proj, etc.).
You can then โ๏ธ freeze the rest of the model and focus your ๐๏ธโโ๏ธ training on the chosen layers.
๐ Results/Evaluation
- Spectrum is competitive with full fine-tuning and beats QLoRA on benchmarks.
- While QLoRA is more memory-efficient on a single GPU, Spectrum shines in distributed training setups.
- Great models trained with Spectrum: Dolphin models, Llama 3.1 Storm, numerous models by VAGO Solutions...
---
For a practical guide, check out the article above.
Full walkthrough on how to get started with Spectrum and TRL for efficient fine-tuning.
๐ ๐ฃ https://huggingface.co/blog/anakin87/spectrum
---
Looking to fine-tune Language Models efficiently and save on computational resources?
One popular method is QLoRa, which quantizes the original model and trains low-rank adapters on top.
It's quite effective and uses less GPU than full fine-tuning.
However, QLoRa applies Low-Rank Adaptation uniformly across the entire model.
What if we could identify the most informative layers and only fine-tune those? ๐ค
This is exactly what Spectrum does! ๐
๐ฌ Spectrum analyzes the weight matrices for all layers in a Language Model and calculates a Signal to Noise Ratio (SNR) for each one.
(It uses Random Matrix Theory and Marchenko-Pastur distribution to distinguish signal from noise.)
๐ฏ Based on a chosen percentage (say, 25%), Spectrum selects the most informative layers of each type (mlp.down_proj, self_attn.o_proj, etc.).
You can then โ๏ธ freeze the rest of the model and focus your ๐๏ธโโ๏ธ training on the chosen layers.
๐ Results/Evaluation
- Spectrum is competitive with full fine-tuning and beats QLoRA on benchmarks.
- While QLoRA is more memory-efficient on a single GPU, Spectrum shines in distributed training setups.
- Great models trained with Spectrum: Dolphin models, Llama 3.1 Storm, numerous models by VAGO Solutions...
---
For a practical guide, check out the article above.
The Forward-Forward Algorithm๐ค
FFA replaces the forward and backward passes in backpropagtion with two forward passes - one with positive (real) data and another with negative data. Each layer has its objective function - to increase or decrease a โgoodness" metric. The positive pass uses real data and adjusts weights to increase โgoodnessโ in every hidden layer. The negative pass does the opposite.
I must say reading&Implementing a godfather paper feels quite fulfilling:)
Thank you Prof. Geoffrey Hinton.
Code: https://github.com/Jaykef/ai-algorithms/blob/main/mnist_the_forward_forward_algor
FFA replaces the forward and backward passes in backpropagtion with two forward passes - one with positive (real) data and another with negative data. Each layer has its objective function - to increase or decrease a โgoodness" metric. The positive pass uses real data and adjusts weights to increase โgoodnessโ in every hidden layer. The negative pass does the opposite.
I must say reading&Implementing a godfather paper feels quite fulfilling:)
Thank you Prof. Geoffrey Hinton.
Code: https://github.com/Jaykef/ai-algorithms/blob/main/mnist_the_forward_forward_algor
Is AIโs impact on elections being overblown? Three researchers think so in this opinion piece published in the MIT Tech Review.
Highlights:
โข โAI is being used to try to influence electoral processes, but these efforts have not been fruitful.โ
โข โWhy were these initial speculations about AI-enabled electoral interference so off (โฆ) ? The short answer: Because they ignored decades of research on the limited influence of mass persuasion campaigns, the complex determinants of voting behaviors, and the indirect and human-mediated causal role of technology.โ
โข โYet we should remember that thereโs a cost to overreaction based on ill-founded assumptions, especially when other critical issues go unaddressed.โ
๐Read more here: https://technologyreview.com/2024/09/03/1103464/ai-impact-elections-overblown/
Highlights:
โข โAI is being used to try to influence electoral processes, but these efforts have not been fruitful.โ
โข โWhy were these initial speculations about AI-enabled electoral interference so off (โฆ) ? The short answer: Because they ignored decades of research on the limited influence of mass persuasion campaigns, the complex determinants of voting behaviors, and the indirect and human-mediated causal role of technology.โ
โข โYet we should remember that thereโs a cost to overreaction based on ill-founded assumptions, especially when other critical issues go unaddressed.โ
๐Read more here: https://technologyreview.com/2024/09/03/1103464/ai-impact-elections-overblown/
๐จ ๐๐๐บ๐ฎ๐ป ๐๐ฒ๐ฒ๐ฑ๐ฏ๐ฎ๐ฐ๐ธ ๐ณ๐ผ๐ฟ ๐๐ ๐๐ฟ๐ฎ๐ถ๐ป๐ถ๐ป๐ด: ๐ก๐ผ๐ ๐๐ต๐ฒ ๐ด๐ผ๐น๐ฑ๐ฒ๐ป ๐ด๐ผ๐ผ๐๐ฒ ๐๐ฒ ๐๐ต๐ผ๐๐ด๐ต๐?
Iโve just read a great paper where Cohere researchers raises significant questions about using Human feedback to evaluate AI language models.
Human feedback is often regarded as the gold standard for judging AI performance, but it turns out, it might be more like fool's gold : the study reveals that our human judgments are easily swayed by factors that have nothing to do with actual AI performance.
๐๐ฒ๐ ๐ถ๐ป๐๐ถ๐ด๐ต๐๐:
๐ง Test several models: Llama-2, Falcon-40B, Cohere Command 6 and 52B ๐ โโ๏ธ Refusing to answer tanks AI ratings more than getting facts wrong. We apparently prefer a wrong answer to no answer!
๐ช Confidence is key (even when it shouldn't be): More assertive AI responses are seen as more factual, even when they're not. This could be pushing AI development in the wrong direction, with systems like RLHF.
๐ญ The assertiveness trap: As AI responses get more confident-sounding, non-expert annotators become less likely to notice when they're wrong or inconsistent.
And a consequence of the above:
๐ ๐ฅ๐๐๐ ๐บ๐ถ๐ด๐ต๐ ๐ฏ๐ฎ๐ฐ๐ธ๐ณ๐ถ๐ฟ๐ฒ: Using human feedback to train AI (Reinforcement Learning from Human Feedback) could accidentally make AI more overconfident and less accurate.
This paper means we need to think carefully about how we evaluate and train AI systems to ensure we're rewarding correctness over apparences of it like confident talk.
โ๏ธ Chatbot Arenaโs ELO leaderboard, based on crowdsourced answers from average joes like you and me, might become completely irrelevant as models will become smarter and smarter.
Read the paper ๐
Human Feedback is not Gold Standard (2309.16349)https://huggingface.co/papers/2309.16349
Iโve just read a great paper where Cohere researchers raises significant questions about using Human feedback to evaluate AI language models.
Human feedback is often regarded as the gold standard for judging AI performance, but it turns out, it might be more like fool's gold : the study reveals that our human judgments are easily swayed by factors that have nothing to do with actual AI performance.
๐๐ฒ๐ ๐ถ๐ป๐๐ถ๐ด๐ต๐๐:
๐ง Test several models: Llama-2, Falcon-40B, Cohere Command 6 and 52B ๐ โโ๏ธ Refusing to answer tanks AI ratings more than getting facts wrong. We apparently prefer a wrong answer to no answer!
๐ช Confidence is key (even when it shouldn't be): More assertive AI responses are seen as more factual, even when they're not. This could be pushing AI development in the wrong direction, with systems like RLHF.
๐ญ The assertiveness trap: As AI responses get more confident-sounding, non-expert annotators become less likely to notice when they're wrong or inconsistent.
And a consequence of the above:
๐ ๐ฅ๐๐๐ ๐บ๐ถ๐ด๐ต๐ ๐ฏ๐ฎ๐ฐ๐ธ๐ณ๐ถ๐ฟ๐ฒ: Using human feedback to train AI (Reinforcement Learning from Human Feedback) could accidentally make AI more overconfident and less accurate.
This paper means we need to think carefully about how we evaluate and train AI systems to ensure we're rewarding correctness over apparences of it like confident talk.
โ๏ธ Chatbot Arenaโs ELO leaderboard, based on crowdsourced answers from average joes like you and me, might become completely irrelevant as models will become smarter and smarter.
Read the paper ๐
Human Feedback is not Gold Standard (2309.16349)https://huggingface.co/papers/2309.16349
Hyperfast Contextual Custom LLM with Agents, Multitokens, Explainable AI, and Distillation https://mltblog.com/4dNPSnB
New additions to this ground-breaking system include multi-token distillation when processing prompts, agents to meet user intent, more NLP, and a command prompt menu accepting both standard prompts and various actions.
I also added several illustrations, featuring xLLM in action with a full session and sample commands to fine-tune in real-time. All the code, input sources (anonymized corporate corpus from fortune 100 company), contextual backend tables including embeddings, are on GitHub. My system has zero weight, no transformer, and no neural network. It relies on explainable AI, does not require training, is fully reproducible, and fits in memory. Yet your prompts can retrieve relevant full text entities from the corpus with no latency โ including URLs, categories, titles, email addresses, and so on โ thanks to well-designed architecture.
Read more, get the code, paper and everything for free, at https://mltblog.com/4dNPSnB
โฆ
New additions to this ground-breaking system include multi-token distillation when processing prompts, agents to meet user intent, more NLP, and a command prompt menu accepting both standard prompts and various actions.
I also added several illustrations, featuring xLLM in action with a full session and sample commands to fine-tune in real-time. All the code, input sources (anonymized corporate corpus from fortune 100 company), contextual backend tables including embeddings, are on GitHub. My system has zero weight, no transformer, and no neural network. It relies on explainable AI, does not require training, is fully reproducible, and fits in memory. Yet your prompts can retrieve relevant full text entities from the corpus with no latency โ including URLs, categories, titles, email addresses, and so on โ thanks to well-designed architecture.
Read more, get the code, paper and everything for free, at https://mltblog.com/4dNPSnB
โฆ
Zero-shot VQA evaluation of Docmatix using LLM - do we need to fine-tune?
While developing Docmatix, we found that fine-tuning Florence-2 performed well on the DocVQA task, but still scored low on the benchmark. To improve the benchmark score, we had to further fine-tune the model on the DocVQA dataset to learn the grammatical style of the benchmark. Interestingly, the human evaluators felt that the additional fine-tuning seemed to perform worse than fine-tuning on Docmatix alone, so we decided to only use the additional fine-tuned model for ablation experiments and publicly release the model fine-tuned on Docmatix alone. Although the answers generated by the model are semantically consistent with the reference answers (as shown in Figure 1), the benchmark scores are low. This raises the question: should we fine-tune the model to improve performance on existing metrics, or should we develop new metrics that are more consistent with human perception?
While developing Docmatix, we found that fine-tuning Florence-2 performed well on the DocVQA task, but still scored low on the benchmark. To improve the benchmark score, we had to further fine-tune the model on the DocVQA dataset to learn the grammatical style of the benchmark. Interestingly, the human evaluators felt that the additional fine-tuning seemed to perform worse than fine-tuning on Docmatix alone, so we decided to only use the additional fine-tuned model for ablation experiments and publicly release the model fine-tuned on Docmatix alone. Although the answers generated by the model are semantically consistent with the reference answers (as shown in Figure 1), the benchmark scores are low. This raises the question: should we fine-tune the model to improve performance on existing metrics, or should we develop new metrics that are more consistent with human perception?
๐
AI Event Scheduler - Streamline event creation with this AI Chrome extension, saving time and reducing manual errors.
๐ Cokeep - Transform bookmarks into collaborative spaces with AI organization, summarization, and team sharing capabilities.
๐จ Crayon AI - Unleash creativity with an all-in-one AI image toolbox, with generation, editing, and optimization for all skill levels.
๐ฅ Tailwind Genie - Generate responsive UI designs with AI, streamlining web development using Tailwind CSS.
๐ค Video Ai Hug - Transform static photos into personalized hugging videos, bringing cherished moments to life.