The A.I Megathread (LLM , GPT , Development)

[bnew]

Spoiler: full text

1/1
[CL] A Philosophical Introduction to Language Models - Part II: The Way Forward
[2405.03207] A Philosophical Introduction to Language Models - Part II: The Way Forward
- This paper explores novel philosophical questions raised by advances in large language models (LLMs) beyond classical debates.

- It examines evidence from causal intervention methods about the nature of LLMs' internal representations and computations.

- It discusses implications of multimodal and modular extensions of LLMs.

- It covers debates about whether LLMs may meet minimal criteria for consciousness.

- It discusses concerns about secrecy and reproducibility in LLM research.

- It discusses whether LLM-like systems may be relevant to modeling human cognition if architecturally constrained.


To post tweets in this format, more info here: https://www.thecoli.com/threads/tips-and-tricks-for-posting-the-coli-megathread.984734/post-52211196

Spoiler: larger images

 

[bnew]

Spoiler: full text

1/1
[CL] A Survey on Large Language Models for Critical Societal Domains: Finance, Healthcare, and Law
[2405.01769] A Survey on Large Language Models for Critical Societal Domains: Finance, Healthcare, and Law
- The paper surveys LLMs in critical societal domains - finance, healthcare, and law.

- These domains rely on professional expertise, have confidential data, multimodal documents, high legal risk, and need for explainability.

- In finance, LLMs assist in analysis, investment, forecasting, but have knowledge gaps. Instruction tuning and retrieval help.

- In healthcare, LLMs aid diagnosis, treatment planning, report generation. Open-sourced medical LLMs are being developed.

- In law, LLMs enable judgment prediction, document analysis, but face data scarcity. Retrieval and debiasing can help.

- Key ethical issues are transparency, justice, non-maleficence. Domain ethics in each field are elaborated.


To post tweets in this format, more info here: https://www.thecoli.com/threads/tips-and-tricks-for-posting-the-coli-megathread.984734/post-52211196

Spoiler: larger images

---

[2405.01769] A Survey on Large Language Models for Critical Societal Domains: Finance, Healthcare, and Law

arxiv.org

Computer Science > Computation and Language​

[Submitted on 2 May 2024]

A Survey on Large Language Models for Critical Societal Domains - Finance, Healthcare, and Law​

Zhiyu Zoey Chen, Jing Ma, Xinlu Zhang, Nan Hao, An Yan, Armineh Nourbakhsh, Xianjun Yang, Julian McAuley, Linda Petzold, William Yang Wang

In the fast-evolving domain of artificial intelligence, large language models (LLMs) such as GPT-3 and GPT-4 are revolutionizing the landscapes of finance, healthcare, and law: domains characterized by their reliance on professional expertise, challenging data acquisition, high-stakes, and stringent regulatory compliance. This survey offers a detailed exploration of the methodologies, applications, challenges, and forward-looking opportunities of LLMs within these high-stakes sectors. We highlight the instrumental role of LLMs in enhancing diagnostic and treatment methodologies in healthcare, innovating financial analytics, and refining legal interpretation and compliance strategies. Moreover, we critically examine the ethics for LLM applications in these fields, pointing out the existing ethical concerns and the need for transparent, fair, and robust AI systems that respect regulatory norms. By presenting a thorough review of current literature and practical applications, we showcase the transformative impact of LLMs, and outline the imperative for interdisciplinary cooperation, methodological advancements, and ethical vigilance. Through this lens, we aim to spark dialogue and inspire future research dedicated to maximizing the benefits of LLMs while mitigating their risks in these precision-dependent sectors. To facilitate future research on LLMs in these critical societal domains, we also initiate a reading list that tracks the latest advancements under this topic, which will be continually updated: \url{this https URL}.

Comments:	35 pages, 6 figures
Subjects:	Computation and Language (cs.CL)
Cite as:	arXiv:2405.01769 [cs.CL]
	(or arXiv:2405.01769v1 [cs.CL] for this version)
	[2405.01769] A Survey on Large Language Models for Critical Societal Domains: Finance, Healthcare, and Law Focus to learn more

Submission history​

From: Zhiyu Chen [view email]
[v1] Thu, 2 May 2024 22:43:02 UTC (5,354 KB)

 

[bnew]

Spoiler: full text

1/1
Google presents Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?

Highlights the risk in introducing new factual knowledge through fine-tuning, which leads to hallucinations

[2405.05904] Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?


To post tweets in this format, more info here: https://www.thecoli.com/threads/tips-and-tricks-for-posting-the-coli-megathread.984734/post-52211196

Spoiler: larger images

 

[bnew]

Spoiler: full text

1/1
From LLMs to Actions: Latent Codes as Bridges in Hierarchical Robot Control

[2405.04798] From LLMs to Actions: Latent Codes as Bridges in Hierarchical Robot Control


To post tweets in this format, more info here: https://www.thecoli.com/threads/tips-and-tricks-for-posting-the-coli-megathread.984734/post-52211196

Spoiler: larger images

 

[bnew]

Spoiler: full text

1/1
Microsoft presents You Only Cache Once: Decoder-Decoder Architectures for Language Models

Substantially reduces GPU memory demands, yet retains global attention capability

repo: unilm/YOCO at master · microsoft/unilm
abs: [2405.05254] You Only Cache Once: Decoder-Decoder Architectures for Language Models


To post tweets in this format, more info here: https://www.thecoli.com/threads/tips-and-tricks-for-posting-the-coli-megathread.984734/post-52211196

Spoiler: larger images

 

[bnew]

Spoiler: full text

1/1
ID-Animator: Zero-Shot Identity-Preserving Human Video Generation

Presents a zero-shot human-video generation approach that can perform personalized video generation given single reference facial image without further training

proj: ID-Animator
abs: [2404.15275] ID-Animator: Zero-Shot Identity-Preserving Human Video Generation


To post tweets in this format, more info here: https://www.thecoli.com/threads/tips-and-tricks-for-posting-the-coli-megathread.984734/post-52211196

 

[bnew]

Spoiler: thread continued

Spoiler: full text

1/3
You can now generate production-ready prompts in the Anthropic Console.

Describe what you want to achieve, and Claude will use prompt engineering techniques like chain-of-thought reasoning to create more effective, precise and reliable prompts.

2/3
Go-to-market platform
@Zoominfo uses Claude to make actionable recommendations and drive value for their customers. Their use of prompt generation helped significantly reduce the time it took to build an MVP of their RAG application, all while improving output quality.

3/3
Our prompt generator also supports dynamic variable insertion, making it easy to test how your prompts perform across different scenarios.

Start generating better prompts today: Anthropic Console


To post tweets in this format, more info here: https://www.thecoli.com/threads/tips-and-tricks-for-posting-the-coli-megathread.984734/post-52211196

Spoiler: larger images

 

[bnew]

Paper: [2404.16821] How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suites

Code: GitHub - OpenGVLab/InternVL: [CVPR 2024 Oral] InternVL Family: A Pioneering Open-Source Alternative to GPT-4V. 接近GPT-4V表现的可商用开源模型

Models: OpenGVLab (OpenGVLab)

Chat demo: InternVL-Chat

Hugging Face demo: InternVL - a Hugging Face Space by OpenGVLab

Abstract:

In this report, we introduce InternVL 1.5, an open-source multimodal large language model (MLLM) to bridge the capability gap between open-source and proprietary commercial models in multimodal understanding. We introduce three simple improvements: (1) Strong Vision Encoder: we explored a continuous learning strategy for the large-scale vision foundation model -- InternViT-6B, boosting its visual understanding capabilities, and making it can be transferred and reused in different LLMs. (2) Dynamic High-Resolution: we divide images into tiles ranging from 1 to 40 of 448×448 pixels according to the aspect ratio and resolution of the input images, which supports up to 4K resolution input. (3) High-Quality Bilingual Dataset: we carefully collected a high-quality bilingual dataset that covers common scenes, document images, and annotated them with English and Chinese question-answer pairs, significantly enhancing performance in OCR- and Chinese-related tasks. We evaluate InternVL 1.5 through a series of benchmarks and comparative studies. Compared to both open-source and proprietary models, InternVL 1.5 shows competitive performance, achieving state-of-the-art results in 8 of 18 benchmarks. Code has been released at this https URL.

 

[bnew]

GitHub - ILikeAI/AlwaysReddy: AlwaysReddy is a LLM voice assistant that is always just a hotkey away.

AlwaysReddy is a LLM voice assistant that is always just a hotkey away. - ILikeAI/AlwaysReddy

github.com

About​

AlwaysReddy is a LLM voice assistant that is always just a hotkey away.

Resources​

Readme

Welcome to AlwaysReddy ​

Hey, I'm Josh, the creator of AlwaysReddy. I am still a little bit of a noob when it comes to programming and I'm really trying to develop my skills over the next year, I'm treating this project as an attempt to better develop my skills, with that in mind I would really appreciate it if you could point out issues and bad practices in my code (of which I'm sure there will be plenty). I would also appreciate if you would make your own improvements to the project so I can learn from your changes. Twitter: https://twitter.com/MindofMachine1

 

[bnew]

 

[bnew]

Spoiler: thread continued

Spoiler: full text

1/8
Exciting new blog -- What’s up with Llama-3?

Since Llama 3’s release, it has quickly jumped to top of the leaderboard. We dive into our data and answer below questions:

- What are users asking? When do users prefer Llama 3?
- How challenging are the prompts?
- Are certain users or prompts over-represented?
- Does Llama 3 have qualitative differences that make users like it?

Key Insights:
1. Llama 3 beats top-tier models on open-ended writing and creative problems but loses a bit on close-ended math and coding problems.

2/8
2. As prompts get challenging*, the gap between Llama 3 against top-tier models becomes larger.

* We define challenging using several criteria like complexity, problem-solving, domain knowledge, and more.

3/8
(Cont'd) We show Llama 3-70b-Instruct's win rate conditioned on hierarchical criteria subsets. Some criteria separate the model's strengths and weaknesses.

4/8
3. Deduplication or outliers do not significantly affect the win rate.

We also sanity-check votes and prompts to avoid certain users being over-represented. Results show that there's no change on Llama 3's win rate before/after.

5/8
4. Qualitatively, we also find Llama 3’s outputs are friendlier and more conversational than other models. These traits appear more often in battles that Llama 3 wins.

Llama 3 also really loves exclamations!

6/8
To conclude, Llama 3 has reached performance on par with top-tier proprietary models in overall use cases. Congrats again to the Llama team
@AIatMeta for such a valuable contribution to the community!

Moving forward, we expect to push new categories to the leaderboard soon based…

7/8
Blog post: What’s up with Llama 3? Arena data analysis | LMSYS Org

Credits to amazing authors!

@lisabdunlap @evan_a_frick @LiTianleli @isaacongjw @profjoeyg @infwinston

8/8
cc @karpathy in case you're still curious


To post tweets in this format, more info here: https://www.thecoli.com/threads/tips-and-tricks-for-posting-the-coli-megathread.984734/post-52211196

Spoiler: larger images

 

[bnew]

 

[bnew]

Spoiler: full text

1/1
OpenAI just announced ChatGPT's new real-time conversational chat.

The model can understand both audio AND video, and can even detect emotion in your voice.

This is insane.


To post tweets in this format, more info here: https://www.thecoli.com/threads/tips-and-tricks-for-posting-the-coli-megathread.984734/post-52211196

Spoiler: larger images

 

[bnew]

Spoiler: thread continued

Spoiler: full text

1/14
live-tweeting our live stream in 1 minute!

2/14
desktop app and new UI

3/14
our new model: GPT-4o, is our best model ever. it is smart, it is fast,it is natively multimodal (!), and…

4/14
it is available to all ChatGPT users, including on the free plan! so far, GPT-4 class models have only been available to people who pay a monthly subscription. this is important to our mission; we want to put great AI tools in the hands of everyone.

5/14
it is a very good model (we had a little fun with the name while testing)

6/14
especially at coding

7/14
in the API, GPT-4o is half the price AND twice as fast as GPT-4-turbo. and 5x rate limits.

8/14
ok now get ready for an amazing demo!!

9/14
check it out:

10/14
and with video mode!!

11/14
real-time voice and video feels so natural; it’s hard to get across by just tweeting. we will roll it out in the coming weeks.

12/14
and for coding!

13/14
audience request to act as a translator

14/14
hope you enjoyed!

the new voice mode will be live in the coming weeks for plus users.

we'll have more stuff to share soon


To post tweets in this format, more info here: https://www.thecoli.com/threads/tips-and-tricks-for-posting-the-coli-megathread.984734/post-52211196

Spoiler: larger images

 

[bnew]