China just wrecked all of American AI. Silicon Valley is in shambles.

[Fani Willis fan]

So cause Trump was elected the Coli is gonna celebrate every US L?

They have developed female traits, from following behind women who naturally will cut off their nose to spite their face.

 

[JT-Money]

Laughing at Silicon Valley douchebags all morning. How all that cheap Indians H1-B's working out for y'all?

Sold off some tech stocks this morning too. Lol

 

[Unemployed GM]

Stereotypical coli post

Never mind. I tried it. It is okay and kind of slow.

 

[Canon]

Never mind. I tried it. It is okay and kind of slow.

its getting pounded(pause) since the news got out

 

[boskey]

Silicon Valley was too focused on changing the outcome of an election and took their eyes off the prize.

Between crypto, the metaverse and politics its been obvious that silicon valley is washed and out of ideas

 

[bnew]

Spoiler: thread continued

Spoiler: full text & large images

1/9
@petesena
DeepSeek R1 blew my mind.
Is it a breakthrough or a Psyop?

If you are an AI nerd & math idiot like me keep reading.

My big takeaway is RL is underappreciated.

It's also a rally cry for open source which pumps me up.

TLDR;
- Reward and rule systems are a HUGE unlock.
- Innovate under constraints: Bigger doesn't mean better.
- Model distillation is a smart and cheap hedge
- Nvidia’s CUDA software an “OS for AI” locks in customers; moats aren’t just hardware
- Execution > vision

Benchmark:

2/9
@petesena
1/ Diving down the rabbit hole of Reddit and armchair experts revealed a lot of trash and speculation. But there were a few rockstar pieces of insight I found on this journey @doodlestein The Short Case for Nvidia Stock put out a great read on this.



3/9
@petesena
2/ As companies start to get drunk on the idea of agents they are failing to realize the rat's nest they will need to untangle. Throw more compute at it isn't the solution in most cases. That's why I love the thinking coming out of companies like Modlee | ML knowledge preservation for the AI era. This whole deepseek stuff (assuming it's not a supercluster psyop) tells me that model distillation and different approaches for training unlock disproportionate results.



4/9
@petesena
3/ Chasing AGI - we're trying to recreate the human brain at a massive scale. Our brains run on something like 20 watts. Everyone is talking about power (electricity + compute), but not enough people are talking about process/approach. While the brain operates on 20 watts, it can perform calculations equivalent to a supercomputer that requires 20 megawatts - making it a million times more energy-efficient. We need smarter ways that aren't just power/compute. Deepseek R1 revealed a kink in Silicon Valley's armor and approach.



5/9
@petesena
4/ DeepSeek vs. The Frontier: Silicon Valley’s Wakeup call.

- Cheaper & Better - ~6M vs GPT-4o 100+M I remember testing Deepseek early and the model Identified itself as OpenAI which points to clear model distillation, why build when you can suck it outta someone else
- Benchmark assassin- They top MATH, Codeforces, and SWE-bench while activating only 37B params
- Hardware constraints = software genius
- Geopolitical jujitsu - US chip bans turned weakness into strength: China’s “innovation under siege” narrative



6/9
@petesena
5/ More proof that accuracy starts with optimization not compute. @tom_doerr "I used DSPy to improve Deepseek V3's accuracy from 14% to 35% when classifying MNIST images, using just the 'light' optimization option."

7/9
@petesena
6/ Stop prompting start programming DSPy has blown my mind. I used to fancy my prompt engineering ability. Then I started using dspy and evals properly. Damn that's an unlock. thx to @tom_doerr for schooling me.



8/9
@petesena
7/ If you found this remotely useful or interesting shoot me a reply or like. I was always scared to go into AI because my 5th-grade math teacher made me feel stupid. Now I struggle my way through it and write about it here - Subscribe

AND: It's working - In under 2 years, I've already built 3 AI companies that do a combined total of 2M in ARR. I'm just getting started. Let's grow together.



9/9
@smdcapital1010
wow




To post tweets in this format, more info here: https://www.thecoli.com/threads/tips-and-tricks-for-posting-the-coli-megathread.984734/post-52211196

GitHub - huggingface/open-r1: Fully open reproduction of DeepSeek-R1

Fully open reproduction of DeepSeek-R1. Contribute to huggingface/open-r1 development by creating an account on GitHub.

github.com

About

Fully open reproduction of DeepSeek-R1

Open R1​

A fully open reproduction of DeepSeek-R1. This repo is a work in progress, let's build it together!

Overview​

The goal of this repo is to build the missing pieces of the R1 pipeline such that everybody can reproduce and build on top of it.

Meet Open R1: The Full Open Reproduction of DeepSeek-R1, Challenging the Status Quo of Existing Proprietary LLMs

Open Source LLM development is going through great change through fully reproducing and open-sourcing DeepSeek-R1, including training data, scripts, etc. Hosted on Hugging Face's platform, this ambitious project is designed to replicate and enhance the R1 pipeline. It emphasizes collaboration...

www.marktechpost.com

Meet Open R1: The Full Open Reproduction of DeepSeek-R1, Challenging the Status Quo of Existing Proprietary LLMs​

By

Asif Razzaq

-

January 26, 2025

Open Source LLM development is going through great change through fully reproducing and open-sourcing DeepSeek-R1, including training data, scripts, etc. Hosted on Hugging Face’s platform, this ambitious project is designed to replicate and enhance the R1 pipeline. It emphasizes collaboration, transparency, and accessibility, enabling researchers and developers worldwide to build on DeepSeek-R1’s foundational work.

What is Open R1?​

Open R1 aims to recreate the DeepSeek-R1 pipeline, an advanced system renowned for its synthetic data generation, reasoning, and reinforcement learning capabilities. This open-source project provides the tools and resources necessary to reproduce the pipeline’s functionalities. The Hugging Face repository will include scripts for training models, evaluating benchmarks, and generating synthetic datasets.

The initiative simplifies the otherwise complex model training and evaluation processes through clear documentation and modular design. By focusing on reproducibility, the Open R1 project invites developers to test, refine, and expand upon its core components.

Key Features of the Open R1 Framework​

1. Training and Fine-Tuning Models: Open R1 includes scripts for fine-tuning models using techniques like Supervised Fine-Tuning (SFT). These scripts are compatible with powerful hardware setups, such as clusters of H100 GPUs, to achieve optimal performance. Fine-tuned models are evaluated on R1 benchmarks to validate their performance.
   
2. Synthetic Data Generation: The project incorporates tools like Distilabel to generate high-quality synthetic datasets. This enables training models that excel in mathematical reasoning and code generation tasks.
   
3. Evaluation: With a specialized evaluation pipeline, Open R1 ensures robust benchmarking against predefined tasks. This provides the effectiveness of models developed using the platform and facilitates improvements based on real-world feedback.
   
4. Pipeline Modularity: The project’s modular design allows researchers to focus on specific components, such as data curation, training, or evaluation. This segmented approach enhances flexibility and encourages community-driven development.
   

Steps in the Open R1 Development Process​

The project roadmap, outlined in its documentation, highlights three key steps:

1. Replication of R1-Distill Models: This involves distilling a high-quality corpus from the original DeepSeek-R1 models. The focus is on creating a robust dataset for further training.
   
2. Development of Pure Reinforcement Learning Pipelines: The next step is to build RL pipelines that emulate DeepSeek’s R1-Zero system. This phase emphasizes the creation of large-scale datasets tailored to advanced reasoning and code-based tasks.
   
3. End-to-End Model Development: The final step demonstrates the pipeline’s capability to transform a base model into an RL-tuned model using multi-stage training processes.
   

Image Source

The Open R1 framework is primarily built in Python, with supporting scripts in Shell and Makefile. Users are encouraged to set up their environments using tools like Conda and install dependencies such as PyTorch and vLLM. The repository provides detailed instructions for configuring systems, including multi-GPU setups, to optimize the pipeline’s performance.

In conclusion, the Open R1 initiative, which offers a fully open reproduction of DeepSeek-R1, will establish the open-source LLM production space at par with large corporations. Since the model capabilities are comparable to those of the biggest proprietary models available, this can be a big win for the open-source community. Also, the project’s emphasis on accessibility ensures that researchers and institutions can contribute to and benefit from this work regardless of their resources. To explore the project further, visit its repository on Hugging Face’s GitHub.

Sources​

• GitHub - huggingface/open-r1: Fully open reproduction of DeepSeek-R1
  
• DeepSeek-R1/DeepSeek_R1.pdf at main · deepseek-ai/DeepSeek-R1
  
• Clem Delangue on LinkedIn: Our science team has started working on fully reproducing and… | 140 comments
  

 

[Spence]

China accomplished this without millions of Indians on H1-B visas?

Nga!

 

[Fani Willis fan]

Never mind. I tried it. It is okay and kind of slow.

They are adjusting to new traffic.

 

[yardman]

It's strange times when capitalists, living in a capitalist society, fear competition.

Competition will always be the fear if the end goal is monopolization, which it usually is. How many startup tech companies get crushed into fine dust after their acquisitions? Competition is just the word given to proles so they don’t raze infrastructure due to disillusionment.​

 

[JT-Money]

DeepSeek Forces a Global Technology Reckoning

DeepSeek Prompts a Reckoning Across Wall Street and Silicon Valley

The fast-growing popularity of the Chinese artificial intelligence software hit shares in tech giants like Nvidia, as Silicon Valley worried about what comes next.

www.nytimes.com

 

[DeuceZ]

Keep telling ourselves we the greatest without investing in the infrastructure needed and act surprised when get lapped up

 

[Spence]

Looks like we about to Liberate China


He was smart releasing this as open source so the country couldn’t claim it and make more money than OpenAI or get banned from Congress like TikTok

Also there’s now no incentive for the CIA to merc his ass since it’s totally out of his hands (Deepseeks creator).

Burn the whole fkn stock market down, it’s been divorced from reality since 2008.

 

[Trojan 24]

So is it better or not? I don't give a fukk about no Indians

 

[Leasy]

I wonder where they get data from

Doesn’t matter they doing it for the good. Open source as it should be.

 

[Robbie3000]

nikkas will say this from a phone or pc that was made in China. While wearing clothes that came from China, while sitting on some shyt that was also made in China, while eating you guessed it. Chinese food.

Some people are so far behind, they actually think they are winning.