The A.I Megathread (LLM , GPT , Development)

[bnew]

Spoiler: full text

1/7
DeepSeek is the level of cracked when a quant finance firm from China is forced to work on AI cuz the government says they're not contributing to society
Imagine if all the perf guys at US quants worked on useful important stuff instead of making markets 0.1 second more efficient

2/7
Liang Wenfeng(deepseek/High flyer founder) was AGI pilled since 2008
Read this interview:
揭秘DeepSeek:一个更极致的中国技术理想主义故事

They also talk about you

3/7
Ye I read it. P cool. Llama 3 paper is better now though maybe

4/7
Renaissance makes too much. Probably higher ROI for them to keep doing quant math

5/7
do you believe that efficient markets are important for facilitating investment in the US? most of my CN friends just invest into real estate which seems like a massive inefficiency in their private sector.

6/7
Is the American financial industry wasting intelligence?

7/7
lol probably it’s because there’s not much money to make in China’s financial market. Derivatives are limited and the equity market never rises. The government always blames quant firms to cover their incompetence. Maybe they are just better off doing AI research …


To post tweets in this format, more info here: https://www.thecoli.com/threads/tips-and-tricks-for-posting-the-coli-megathread.984734/post-52211196

Spoiler: larger images

Spoiler: full text

1/11
Exciting news! DeepSeek API now launches context caching on disk, with no code changes required! This new feature automatically caches frequently referenced contexts on distributed storage, slashing API costs by up to 90%. For a 128K prompt with high reference, the first token latency is cut from 13s to just 500ms.

Benefit Scenarios:
- Multi-turn conversations where subsequent rounds hit the previous context cache.
- Data analysis with recurring queries on the same documents/files.
- Code analysis/debugging with repeated repository references.

DeepSeek API disk caching is now live with unlimited concurrency. Billing is automatic based on actual cache hits. Learn more at DeepSeek API introduces Context Caching on Disk, cutting prices by an order of magnitude | DeepSeek API Docs
/search?q=#DeepSeek /search?q=#ContextCaching

2/11
It seems that a response issue has appeared after the update. 500s is crazy.

3/11
The issue has been fixed, and service is now restored.

4/11
interesting feature for cost savings

i've started using gpt4 mini to reduce costs but would love to stick with more expensive models on http://microlaunch.net

eventually w/ context caching

does deepseek also handle prompt compression?

5/11
This is awesome

6/11
I you (I'm literally just a whale)

7/11
The DeepSeek V2 and Coder V2 OpenRouter's API version are running the update 0628?

8/11
我没有中国手机号，我该怎么使用它呢？

9/11
wait so... this just automatically happens if you're using the deepseek models now? only through deepseek, or through openrouter/deekseek too? i'd love more clarification - amazing nonetheless!

10/11
国产之光！为你们感到骄傲！

11/11
This is amazing. Will it work for openrouter also?


To post tweets in this format, more info here: https://www.thecoli.com/threads/tips-and-tricks-for-posting-the-coli-megathread.984734/post-52211196

Spoiler: larger images

 

[bnew]

 

[bnew]

[2310.18318] OpenCog Hyperon: A Framework for AGI at the Human Level and Beyond

[Submitted on 19 Sep 2023]

OpenCog Hyperon: A Framework for AGI at the Human Level and Beyond​

Ben Goertzel, Vitaly Bogdanov, Michael Duncan, Deborah Duong, Zarathustra Goertzel, Jan Horlings, Matthew Ikle', Lucius Greg Meredith, Alexey Potapov, Andre' Luiz de Senna, Hedra Seid Andres Suarez, Adam Vandervorst, Robert Werko

An introduction to the OpenCog Hyperon framework for Artificiai General Intelligence is presented. Hyperon is a new, mostly from-the-ground-up rewrite/redesign of the OpenCog AGI framework, based on similar conceptual and cognitive principles to the previous OpenCog version, but incorporating a variety of new ideas at the mathematical, software architecture and AI-algorithm level. This review lightly summarizes: 1) some of the history behind OpenCog and Hyperon, 2) the core structures and processes underlying Hyperon as a software system, 3) the integration of this software system with the SingularityNET ecosystem's decentralized infrastructure, 4) the cognitive model(s) being experimentally pursued within Hyperon on the hopeful path to advanced AGI, 5) the prospects seen for advanced aspects like reflective self-modification and self-improvement of the codebase, 6) the tentative development roadmap and various challenges expected to be faced, 7) the thinking of the Hyperon team regarding how to guide this sort of work in a beneficial direction ... and gives links and references for readers who wish to delve further into any of these aspects.

Subjects:	Artificial Intelligence (cs.AI)
Cite as:	arXiv:2310.18318 [cs.AI]
	(or arXiv:2310.18318v1 [cs.AI] for this version)
	[2310.18318] OpenCog Hyperon: A Framework for AGI at the Human Level and Beyond

Submission history​

From: Benjamin Goertzel [view email]

[v1] Tue, 19 Sep 2023 23:25:09 UTC (1,856 KB)

https://arxiv.org/pdf/2310.18318

 

[bnew]

[1901.03775] Creative AI Through Evolutionary Computation

[Submitted on 12 Jan 2019 (v1), last revised 22 Feb 2020 (this version, v2)]

Creative AI Through Evolutionary Computation​

Risto Miikkulainen

The main power of artificial intelligence is not in modeling what we already know, but in creating solutions that are new. Such solutions exist in extremely large, high-dimensional, and complex search spaces. Population-based search techniques, i.e. variants of evolutionary computation, are well suited to finding them. These techniques are also well positioned to take advantage of large-scale parallel computing resources, making creative AI through evolutionary computation the likely "next deep learning".

Subjects:	Neural and Evolutionary Computing (cs.NE); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
Cite as:	arXiv:1901.03775 [cs.NE]
	(or arXiv:1901.03775v2 [cs.NE] for this version)
	[1901.03775] Creative AI Through Evolutionary Computation

Journal reference:	In Banzhaf et al. (editors), Evolution in Action---Past, Present and Future. New York: Springer. 2020

Submission history​

From: Risto Miikkulainen [view email]

[v1] Sat, 12 Jan 2019 00:26:13 UTC (694 KB)

[v2] Sat, 22 Feb 2020 23:15:46 UTC (694 KB)

https://arxiv.org/pdf/1901.03775

 

[bnew]

AI achieves silver-medal standard solving International Mathematical Olympiad problems

Breakthrough models AlphaProof and AlphaGeometry 2 solve advanced reasoning problems in mathematics

deepmind.google

AI achieves silver-medal standard solving International Mathematical Olympiad problems​

Published 25 July 2024 Authors

AlphaProof and AlphaGeometry teams

Breakthrough models AlphaProof and AlphaGeometry 2 solve advanced reasoning problems in mathematics

Artificial general intelligence (AGI) with advanced mathematical reasoning has the potential to unlock new frontiers in science and technology.

We’ve made great progress building AI systems that help mathematicians discover new insights, novel algorithms and answers to open problems. But current AI systems still struggle with solving general math problems because of limitations in reasoning skills and training data.

Today, we present AlphaProof, a new reinforcement-learning based system for formal math reasoning, and AlphaGeometry 2, an improved version of our geometry-solving system. Together, these systems solved four out of six problems from this year’s International Mathematical Olympiad (IMO), achieving the same level as a silver medalist in the competition for the first time.

​

Breakthrough AI performance solving complex math problems​

The IMO is the oldest, largest and most prestigious competition for young mathematicians, held annually since 1959.

Each year, elite pre-college mathematicians train, sometimes for thousands of hours, to solve six exceptionally difficult problems in algebra, combinatorics, geometry and number theory. Many of the winners of the Fields Medal, one of the highest honors for mathematicians, have represented their country at the IMO.

More recently, the annual IMO competition has also become widely recognised as a grand challenge in machine learning and an aspirational benchmark for measuring an AI system’s advanced mathematical reasoning capabilities.

This year, we applied our combined AI system to the competition problems, provided by the IMO organizers. Our solutions were scored according to the IMO’s point-awarding rules by prominent mathematicians Prof Sir Timothy Gowers, an IMO gold medalist and Fields Medal winner, and Dr Joseph Myers, a two-time IMO gold medalist and Chair of the IMO 2024 Problem Selection Committee.

“

The fact that the program can come up with a non-obvious construction like this is very impressive, and well beyond what I thought was state of the art.

Prof Sir Timothy Gowers,

IMO gold medalist and Fields Medal winner

First, the problems were manually translated into formal mathematical language for our systems to understand. In the official competition, students submit answers in two sessions of 4.5 hours each. Our systems solved one problem within minutes and took up to three days to solve the others.

AlphaProof solved two algebra problems and one number theory problem by determining the answer and proving it was correct. This included the hardest problem in the competition, solved by only five contestants at this year’s IMO. AlphaGeometry 2 proved the geometry problem, while the two combinatorics problems remained unsolved.

• 
  See our system's IMO 2024 solutions

Each of the six problems can earn seven points, with a total maximum of 42. Our system achieved a final score of 28 points, earning a perfect score on each problem solved — equivalent to the top end of the silver-medal category. This year, the gold-medal threshold starts at 29 points, and was achieved by 58 of 609 contestants at the official competition.

Graph showing performance of our AI system relative to human competitors at IMO 2024. We earned 28 out of 42 total points, achieving the same level as a silver medalist in the competition.

​

AlphaProof: a formal approach to reasoning​

AlphaProof is a system that trains itself to prove mathematical statements in the formal language Lean. It couples a pre-trained language model with the AlphaZero reinforcement learning algorithm, which previously taught itself how to master the games of chess, shogi and Go.

Formal languages offer the critical advantage that proofs involving mathematical reasoning can be formally verified for correctness. Their use in machine learning has, however, previously been constrained by the very limited amount of human-written data available.

In contrast, natural language based approaches can hallucinate plausible but incorrect intermediate reasoning steps and solutions, despite having access to orders of magnitudes more data. We established a bridge between these two complementary spheres by fine-tuning a Gemini model to automatically translate natural language problem statements into formal statements, creating a large library of formal problems of varying difficulty.

When presented with a problem, AlphaProof generates solution candidates and then proves or disproves them by searching over possible proof steps in Lean. Each proof that was found and verified is used to reinforce AlphaProof’s language model, enhancing its ability to solve subsequent, more challenging problems.

We trained AlphaProof for the IMO by proving or disproving millions of problems, covering a wide range of difficulties and mathematical topic areas over a period of weeks leading up to the competition. The training loop was also applied during the contest, reinforcing proofs of self-generated variations of the contest problems until a full solution could be found.

Process infographic of AlphaProof’s reinforcement learning training loop: Around one million informal math problems are translated into a formal math language by a formalizer network. Then a solver network searches for proofs or disproofs of the problems, progressively training itself via the AlphaZero algorithm to solve more challenging problems.

​

A more competitive AlphaGeometry 2​

AlphaGeometry 2 is a significantly improved version of AlphaGeometry. It’s a neuro-symbolic hybrid system in which the language model was based on Gemini and trained from scratch on an order of magnitude more synthetic data than its predecessor. This helped the model tackle much more challenging geometry problems, including problems about movements of objects and equations of angles, ratio or distances.

AlphaGeometry 2 employs a symbolic engine that is two orders of magnitude faster than its predecessor. When presented with a new problem, a novel knowledge-sharing mechanism is used to enable advanced combinations of different search trees to tackle more complex problems.

Before this year’s competition, AlphaGeometry 2 could solve 83% of all historical IMO geometry problems from the past 25 years, compared to the 53% rate achieved by its predecessor. For IMO 2024, AlphaGeometry 2 solved Problem 4 within 19 seconds after receiving its formalization.

Illustration of Problem 4, which asks to prove the sum of ∠KIL and ∠XPY equals 180°. AlphaGeometry 2 proposed to construct E, a point on the line BI so that ∠AEB = 90°. Point E helps give purpose to the midpoint L of AB, creating many pairs of similar triangles such as ABE ~ YBI and ALE ~ IPC needed to prove the conclusion.

​

New frontiers in mathematical reasoning​

As part of our IMO work, we also experimented with a natural language reasoning system, built upon Gemini and our latest research to enable advanced problem-solving skills. This system doesn’t require the problems to be translated into a formal language and could be combined with other AI systems. We also tested this approach on this year’s IMO problems and the results showed great promise.

Our teams are continuing to explore multiple AI approaches for advancing mathematical reasoning and plan to release more technical details on AlphaProof soon.

We’re excited for a future in which mathematicians work with AI tools to explore hypotheses, try bold new approaches to solving long-standing problems and quickly complete time-consuming elements of proofs — and where AI systems like Gemini become more capable at math and broader reasoning.

​

Acknowledgements​

We thank the International Mathematical Olympiad organization for their support.

AlphaProof development was led by Thomas Hubert, Rishi Mehta and Laurent Sartran; AlphaGeometry 2 and natural language reasoning efforts were led by Thang Luong.

 

[bnew]

https://old.reddit.com/r/FluxAI/

Flux Examples

Examples of ComfyUI workflows

comfyanonymous.github.io

black-forest-labs/flux-schnell – Run with an API on Replicate

The fastest image generation model tailored for local development and personal use

replicate.com

 

[bnew]

 

[bnew]

 

[bnew]

 

[bnew]

OpenAI's brain drain isn't a great look for Sam Altman

OpenAI is losing key members like cofounder John Schulman to rivals, adding to challenges in the AI market and trust issues for CEO Sam Altman.

www.businessinsider.com

OpenAI's brain drain isn't a great look for Sam Altman​

Jordan Hart

OpenAI will have to weather the storms of being the first company to take generative AI mainstream. Justin Sullivan/Getty Images;Chelsea Jia Feng/BI

• 
  OpenAI is losing key members, including cofounder John Schulman, who left for Anthropic.
  
• 
  Despite leading the market with ChatGPT, OpenAI faces big challenges in the fledgling AI world.
  
• 
  Trust issues and whistleblower complaints add to Sam Altman's obstacles as CEO.
  

OpenAI is losing key members during a pivotal time in the artificial intelligence market, and it's not a good look.

On Monday, cofounder John Schulman announced that he would be leaving OpenAI to work at rival Anthropic. He joins two other high-level executives — though president Greg Brockman said he's taking an extended leave of absence — and several former employees in what appears to be an exodus from the ChatGPT maker helmed by Sam Altman.

"It's not good at all for OpenAI," said tech analyst Jacob Bourne at Emarketer, a sister company to Business Insider.

Although OpenAI got ahead in the AI arms race when it released its chatbot in a surprise move in November 2022, being the first may not be enough to keep it at the top of the leaderboard as other, bigger companies build and release their own AI and key executives depart.

"OpenAI has no 'moat,'" Mike Gualtieri, vice president and principal analyst at Forrester, said.

In business, a moat refers to a significant advantage that keeps a company more or less untouchable from its rivals and helps it maintain its market share.

Gualtieri told BI that Big Tech companies, Google and Meta in particular, already had generative AI tech at the same time as OpenAI.

"They were just afraid to release because hallucinations, etc, could impact their reputation and business," Gualtieri said.

Just last week, Meta released a statement addressing a hallucination that prompted its MetaAI chatbot to say that the July 13 assassination attempt of former President Donald Trump didn't happen. The story drew a lot of attention and criticism.

"I think we're going to see more of this kind of scrutiny, and it's not going to be just directed at OpenAI," Bourne said

OpenAI — and Sam Altman — are also under a lot of other scrutiny. On July 1, whistleblowers at OpenAI contacted the Securities and Exchange Commission, calling for it to investigate the company for rule violations around NDAs. Weeks before that, nine current and former OpenAI employees signed an open letter pointing out the risks of generative AI. And the company's management has been seen to be split between pressing ahead with AI development and having a more cautious approach.

"I decided to leave OpenAI because I lost hope that they would act responsibly, particularly as they pursue artificial general intelligence," Daniel Kokotajlo, a former OpenAI employee who signed the letter, previously said in a statement.

Tech companies are spending heavily on AI endeavors, but trust remains one of the key factors in how much their investments will pay off.

Emarketer's Bourne said, "It's kind of this perfect storm for the emergence of this kind of concern around profits over safety that we're seeing."

And, Bourne said, as a young company with an "unusual government structure," OpenAI will continue to be under a magnifying glass — possibly even more than well-established rivals.

Meantime, it looks like OpenAI's brain drain to competitors could put the company at a further disadvantage despite its first-mover advantage and Apple partnership.

 

[bnew]

 

[bnew]

Spoiler: thread continued

Spoiler: full text

1/5
Graph RAG is hot!
Our "Graph Chain-of-Thought: Augmenting Large Language Models by Reasoning on Graphs" has been accepted to ACL 2024.

️GRBench: A new benchmark for graph RAG research.
️Graph CoT: An iterative framework to let LLM explore on graph environments. /search?q=#graph /search?q=#LLM

2/5
paper: [2404.07103] Graph Chain-of-Thought: Augmenting Large Language Models by Reasoning on Graphs
data: PeterJinGo/GRBench · Datasets at Hugging Face
code: GitHub - PeterGriffinJin/Graph-CoT: Graph Chain-of-Thought: Augmenting Large Language Models by Reasoning on Graphs (ACL 2024)

3/5
This is a joint work with @ChulinXie, Jiawei Zhang, Kashob Roy, @yuz9yuz, Suhang Wang, @yumeng0818 and Jiawei Han @dmguiuc.

4/5
Finally someone does what I am thinking about

5/5
Check out this one related to your paper:


To post tweets in this format, more info here: https://www.thecoli.com/threads/tips-and-tricks-for-posting-the-coli-megathread.984734/post-52211196

Spoiler: larger images

 

[bnew]

Spoiler: thread continued

Spoiler: full text

1/11
>50% chance myself and @csahil28 will release an LLM that smashes current benchmarks, using a new reasoning technique

Stay tuned

2/11
New reasoning technique?

3/11
Sort of an extension to chain-of-thought, will share more soon, whether we succeed or fail

4/11
sir

5/11
sir

6/11
yo is this fr????

7/11
Still making sure, but it seems so. As of now, we're seeing improvements on reasoning tasks, but slight regressions elsewhere. We think we have a fix. If so, it's real.

8/11
Sahil if you drop this in the next 2-3 I'll add to the newsletter

9/11
We're still on mid-size LLM runs, it's likely gonna be be another week or so

10/11
<|im_start|>reasoning

11/11
lol not close, but not far off


To post tweets in this format, more info here: https://www.thecoli.com/threads/tips-and-tricks-for-posting-the-coli-megathread.984734/post-52211196

Spoiler: larger images

 

[bnew]

Spoiler: thread continued

Spoiler: full text

1/7
Holy shyt.

I think I may have proven reasoning-based approaches' efficacy, in Claude.

So, I found a riddle that Claude couldn't solve.

We establish the baseline (vanilla) response.

2/7
Claude took a crack at it vanilla, and reasoned that it was BREW.

Deductive reasoning? BREW

Inductive? BREW

Analogical? Abductive?

BREW
BREW

3/7
Lateral?

LOSE

Finally, a different answer.

Still wrong, but different.

That should be expected, though, because lateral thinking involves approaching the problem from a different angle.

But that means it *did* use lateral thinking to approach the problem.

(Minor) Success!

4/7
We proceed with a lot of different reasoning approaches, and a lot of wrong answers.

HOPE? Nope.

WISH? You wish.

BREW, a dozen more times? Still no.

5/7
Finally, pattern recognition gets us remarkably close to the correct answer.

DEBT is close, but not quite it.

But, again, it used the reasoning type successfully to give us a novel answer that is different from the baseline/vanilla response.

6/7
At a certain point it started cycling back through reasoning types.

And, success!

Analogical reasoning actually presented the right answer; something no other approach had done.

Keep in mind, I'm continuing the session with the exact same text every time, no hints.

7/7
DIET.

It even got one of the hints right, which no other approach managed to do.

In the end?

It still chose BREW.

But this experiment proved that results and output are dramatically different based on the reasoning approach(es) you instruct.

Keep it in mind, prompt pals.


To post tweets in this format, more info here: https://www.thecoli.com/threads/tips-and-tricks-for-posting-the-coli-megathread.984734/post-52211196

Spoiler: larger images

 

[bnew]

Spoiler: full text

1/1
Open Source Today (2024-08-09): Tongyi Qianwen Releases Qwen2-Math for Advanced Math Reasoning

Open Source Today (2024-08-09): Tongyi Qianwen Releases Qwen2-Math for Advanced Math Reasoning

/search?q=#AI /search?q=#LLM /search?q=#OpenSource /search?q=#Qwen2Math /search?q=#Qwen2 /search?q=#Qwen /search?q=#TTS /search?q=#frameworks


To post tweets in this format, more info here: https://www.thecoli.com/threads/tips-and-tricks-for-posting-the-coli-megathread.984734/post-52211196

Spoiler: larger images

Spoiler: thread continued

Spoiler: full text

1/6
CONGRATS to @Alibaba_Qwen team on Qwen2-Math-72B outperforming GPT-4o, Claude-3.5-Sonnet, Gemini-1.5-Pro, Llama-3.1-405B on a series of math benchmarks

2/6
So beautiful -
Qwen2-Math is capable of solving simple math competition problems.

3/6
From its license doc

"If you are commercially using the Materials, and your product or service has more than 100 million monthly active users, You shall request a license from Us. You cannot exercise your rights under this Agreement without our express authorization."

4/6
Some nice details on the training data filtration

5/6
transformers>=4.40.0 for Qwen2-Math models. The latest version is recommended.

6/6
Yeah, but it's still unreliable at math, so of limited use, and it can't do anything else. For example, it can't even answer basic questions about world knowledge or write a story (writes them in outline form, then gives a conclusion). If not part of a MOA I don't see the point.


To post tweets in this format, more info here: https://www.thecoli.com/threads/tips-and-tricks-for-posting-the-coli-megathread.984734/post-52211196

Spoiler: larger images

Spoiler: full text

1/11
Today we release a new model series for math-specific language models, Qwen2-Math, which is based on Qwen2. The flagship model, Qwen2-Math-72B-Instruct, outperforms proprietary models, including GPT-4o and Claude 3.5, in math related downstream tasks!

Feel free to check our blog for more information:
Introducing Qwen2-Math

HF Collections： Qwen2-Math - a Qwen Collection

Github: GitHub - QwenLM/Qwen2-Math: A series of math-specific large language models of our Qwen2 series.

2/11
We evaluate our math-specific models on a series of math benchmarks. The results below demonstrate that our largest math-specific model Qwen2-Math-72B-Instruct outperforms the state-of-the-art models, including GPT-4o, Claude-3.5-Sonnet, Gemini-1.5-Pro, Llama-3.1-405B.

3/11
Congrats Qwen team!
We're going to cover this live in 1 hour on the @thursdai_pod show!

4/11
You guys shipped real

5/11
@ollama

6/11
What if we could get this to 95% What would that unlock?

7/11
@bindureddy can we get this model in abacus?

8/11
man this post needs more attention.

all the hype is going to strawberry hype but this is what we like to see.

good job Qwen Team!

9/11
Wow!!

10/11
@altryne just in time for best day ever, well done @JustinLin610

11/11
Congrats! What's RM@N?


To post tweets in this format, more info here: https://www.thecoli.com/threads/tips-and-tricks-for-posting-the-coli-megathread.984734/post-52211196

Spoiler: larger images

Spoiler: full text

1/5
Qwen2-Math

huggingface: Qwen2-Math - a Qwen Collection

The flagship model, Qwen2-Math-72B-Instruct, outperforms proprietary models, including GPT-4o and Claude 3.5, in math related downstream tasks

2/5
Qwen2 Math 1.5B and 7B version now available in ModelBox Inference. Try now :

Qwen2 Math 7B Instruct Playground | ModelBox

3/5
now everyone can freely try it out here
Qwen2 Math 7B Instruct Playground | ModelBox

4/5
Finally, a math-focused LLM that can give proprietary models a run for their money!

5/5
Brought to you by Nvidia Graphworks


To post tweets in this format, more info here: https://www.thecoli.com/threads/tips-and-tricks-for-posting-the-coli-megathread.984734/post-52211196

Spoiler: larger images