HF-hub

A list of the missing papers in the proceedings of the ACL 2024.

ACL 2024: The Missing Papers

Apparently, some papers from the ACL 2024 are still not listed in the ACL Anthology. While this issue will hopefully be fixed soon, we should give those papers additional spotlight.

Some of my favorites:

1. Dolma is an English corpus that encompasses 3 trillion tokens. Additionally, it is accompanied by an exceptional software package that consdierably advances the state-of-the-art in preparing data for LLM pretraining. (Source: I am currently using Dolma.)
Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Research (2402.00159)

2. In the paper "Same Task, More Tokens: the Impact of Input Length on
the Reasoning Performance of Large Language Models", the authors show how extending the context length impacts an LLM's reasoning performance. I asked myself a similar question a few months ago, and therefore this paper is highly interesting to me.
Same Task, More Tokens: the Impact of Input Length on the Reasoning Performance of Large Language Models (2402.14848)

This was brought to my attention through a Linkedin post by @ShayeghB, who is also affected:
Ensemble-Based Unsupervised Discontinuous Constituency Parsing by Tree Averaging (2403.00143)

View all the missing papers here:
https://theshayegh.github.io/ACL2024MissingPapers/

ACL2024MissingPapers

ACL 2024 Missing Papers

v2ray/deepgelbooru · Hugging Face

v2ray/deepgelbooru

A Danbooru tag image tagger, maybe better than WD14 at some images.
Training code, inference code, dataset included.
:3
https://huggingface.co/v2ray/deepgelbooru

huggingface.co

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

I can't believe this... Phi-3.5-mini (3.8B) running in-browser at ~90 tokens/second on WebGPU w/ Transformers.js and ONNX Runtime Web! 🤯 Since everything runs 100% locally, no messages are sent to a server — a huge win for privacy!
- 🤗 Demo:
webml-community/phi-3.5-webgpu

- 🧑‍💻 Source code: https://github.com/huggingface/transformers.js-examples/tree/main/phi-3.5-webgpu

transformers.js-examples/phi-3.5-webgpu at main · huggingface/transformers.js-examples

A collection of 🤗 Transformers.js demos and example applications - huggingface/transformers.js-examples

You can now use DoRA for your embedding layers!

PR: https://github.com/huggingface/peft/pull/2006

I have documented my journey of this specific PR in a blog post for everyone to read. The highlight of the PR was when the first author of DoRA reviewed my code.

Blog Post: https://huggingface.co/blog/ariG23498/peft-dora

Huge thanks to @BenjaminB for all the help I needed.

[Add] DoRA Embedding by ariG23498 · Pull Request #2006 · huggingface/peft

This PR adds DoRA for the embedding layer.
TODOs:

Check whether this implementation is correct
Write tests for the implementation

@BenjaminBossan can you give me an initial review to set the to...

05:10 · Aug 24, 2024 · Sat

🌐 Check out the new dataset sourced from Fishki.net, one of the popular entertainment and news portals in the Russian Internet, known for its diverse content including humor, interesting facts, and viral stories -
nyuuzyou/fishkinet-posts
.

📊 Dataset highlights:
- 369,180 posts
- Includes original posts with titles, content, images, and metadata
- Each entry contains URL, title, author, date, tags, content, and image URLs
- Primarily in Russian language
- Covers a wide range of topics in entertainment, news, and social media content
- Spans nearly two decades of posts, likely from early 2000s to 2024
- Dedicated to public domain under Creative Commons Zero (CC0) license

Alan Turing's mind-bender: "Can machines think?" in its glorified form. This 74yr old paper laid the foundation for how we think about AI and machine intelligence today. The level of detail, clarity and foresight is just phenomenal - he was way ahead of his time 🧠🤖

Original copy: https://archive.org/details/MIND--COMPUTING-MACHINERY-AND-INTELLIGENCE

Internet Archive

I.—COMPUTING MACHINERY AND INTELLIGENCE : A.M Turing : Free Download, Borrow, and Streaming : Internet Archive

I.—COMPUTING MACHINERY ANDINTELLIGENCE1. The Imitation Game.I PROPOSE to consider the question, ' Can machines think ? 'This should begin with definitions of...

12:03 · Aug 22, 2024 · Thu

g/ - /ldg/ - Local Diffusion General - Technology

>Beginner UI
EasyDiffusion: https://easydiffusion.github.io
Fooocus: https://github.com/lllyasviel/fooocus
Metastable: https://metastable.studio

>Advanced UI
Automatic1111: https://github.com/automatic1111/stable-diffusion-webui
ComfyUI: https://github.com/comfyanonymous/ComfyUI
InvokeAI: https://github.com/invoke-ai/InvokeAI
SD.Next: https://github.com/vladmandic/automatic
SwarmUI: https://github.com/mcmonkeyprojects/SwarmUI

GitHub - lllyasviel/Fooocus: Focus on prompting and generating

Focus on prompting and generating. Contribute to lllyasviel/Fooocus development by creating an account on GitHub.

Introducing HelpingAI2-9B, an emotionally intelligent LLM.
Model Link :
OEvortex/HelpingAI2-9B

Demo Link:
Abhaykoul/HelpingAI2

This model is part of the innovative HelpingAI series and it stands out for its ability to engage users with emotional understanding.

Key Features:
-----------------

* It gets 95.89 score on EQ Bench greather than all top notch LLMs, reflecting advanced emotional recognition.
* It gives responses in empathetic and supportive manner.

Must try our demo:
Abhaykoul/HelpingAI2

NEW math-instruct model + dataset!

ValiantLabs/Llama3.1-8B-Cobalt
is our new math-instruct model.
Trained using a synthetic math-instruct dataset generated with Llama 3.1 405b. Find the dataset here:
sequelbox/Polytope

More to come soon :)

Supercool Weekend Read🤖
Nvidia researchers achieved SOTA LLM compression metrics using pruning and knowledge distillation techniques.

Details on Techniques (Simplified):
They started off with a large pre-trained language model (15B params), then:

1. Estimated the importance of different parts of the model (neurons, attention heads, layers) using activation-based metrics on a small calibration dataset.

What Happens When RAG System Become Fully Vision-Language Model-Based?
HF Demo:
bokesyo/MiniCPMV-RAG-PDFQA

Multimodal Dense Retriever:
RhapsodyAI/minicpm-visual-embedding-v0

Generation Model:
openbmb/MiniCPM-V-2_6

Github: https://github.com/RhapsodyAILab/MiniCPM-V-Embedding-v0-Train

The Vision-Language Model Dense Retriever MiniCPM-Visual-Embedding-v0 reads PDFs directly -- no OCR required. With strong OCR capability and visual understanding capability, it generates multimodal dense representations, allowing you to build and search through your personal library with ease.

Ask a question, it retrieves the most relevant pages. Then, MiniCPM-V-2.6 provides answers based on the retrieved pages, with strong multi-image understanding capabilities.

Whether you’re working with a visually-intensive or text-oriented PDF, it helps you quickly find the information you need. You can also build a personal library with it.

It operates just like a human: reading, storing, retrieving, and answering with full visual comprehension.

Currently, the online demo supports PDFs with up to 50 pages due to GPU time limits. For longer PDFs or entire books, you can deploy it on your own machine.
https://cdn-uploads.huggingface.co/production/uploads/6415818a986557e8cac252bf/sjtQD7CFgox46h9EVHCG_.png

GitHub - RhapsodyAILab/MiniCPM-V-Embedding

Contribute to RhapsodyAILab/MiniCPM-V-Embedding development by creating an account on GitHub.

So turns out I've been spreading a bit of misinformation when it comes to imatrix in llama.cpp

It starts true; imatrix runs the model against a corpus of text and tracks the activation of weights to determine which are most important

However what the quantization then does with that information is where I was wrong.

I think I made the accidental connection between imatrix and exllamav2's measuring, where ExLlamaV2 decides how many bits to assign to which weight depending on the goal BPW

Instead, what llama.cpp with imatrix does is it attempts to select a scale for a quantization block that most accurately returns the important weights to their original values, ie minimizing the dequantization error based on the importance of activations

The mildly surprising part is that it actually just does a relatively brute force search, it picks a bunch of scales and tries each and sees which one results in the minimum error for weights deemed important in the group

But yeah, turns out, the quantization scheme is always the same, it's just that the scaling has a bit more logic to it when you use imatrix

Huge shoutout to @compilade for helping me wrap my head around it - feel free to add/correct as well if I've messed something up

How good are you at spotting AI-generated images?

Find out by playing Fake Insects 🐞 a Game where you need to identify which insects are fake (AI generated). Good luck & share your best score in the comments!

victor/fake-insects

I'm excited to share a really cool milestone in my AI/LLM journey.

Brief backstory: Before diving into AI, I spent over a decade working in ecological fields such as the conservation corps, biodynamic farming, and natural habitat restoration. This background instilled in me a deep concern about the environmental impact of scaling AI without sustainable practices.

Driven by this concern, I've spent months planning and experimenting to make my AI work more eco-friendly. I'm thrilled to announce that I've successfully transitioned my entire operation to run on 100% sustainable solar power!

🚀 We’re excited to launch Ghost 8B Beta (1608), a top-performing language model with unmatched multilingual support and cost efficiency.

Key Highlights:
- Superior Performance: Outperforms Llama 3.1 8B Instruct, GPT-3.5 Turbo, Claude 3 Opus, GPT-4, and more in winrate scores.
- Expanded Language Support: Now supports 16 languages, including English, Vietnamese, Spanish, Chinese, and more.
- Enhanced Capabilities: Improved math, reasoning, and instruction-following for better task handling.

🔔 Release: small-text v1.4.1

The new release contains some smaller bugfixes. Check it out!

Github: https://github.com/webis-de/small-text
Paper:
Small-Text: Active Learning for Text Classification in Python (2107.10314)

GitHub - webis-de/small-text: Active Learning for Text Classification in Python

Active Learning for Text Classification in Python. Contribute to webis-de/small-text development by creating an account on GitHub.

Put together a small repo showing how to go from making your own fine-tuning dataset w/ services like Groq & Together to publishing that model on ollama.

In my case I fine-tuned SmolLM-360M to be a better assistant for my Pi-Card (previous post) project.

Check it out!
https://github.com/nkasmanoff/ft-flow

GitHub - nkasmanoff/ft-flow: Synthetic data to inference for LLM finetuning

Synthetic data to inference for LLM finetuning. Contribute to nkasmanoff/ft-flow development by creating an account on GitHub.

ResShift 1-Click Windows, RunPod, Massed Compute, Kaggle Installers with Amazing Gradio APP and Batch Image Processing. ResShift is Efficient Diffusion Model for Image Super-resolution by Residual Shifting (NeurIPS 2023, Spotlight).

Official Repo : https://github.com/zsyOAOA/ResShift

I have developed a very advanced Gradio APP.