The A.I Megathread (LLM , GPT , Development)

[bnew]

White House says no need to restrict 'open-source' artificial intelligence — at least for now

The White House is coming out in favor of “open-source” artificial intelligence technology, arguing in a report Tuesday that there’s no need right now for restrictions on companies making key components of their powerful AI systems widely available.

apnews.com

​

White House says no need to restrict ‘open-source’ artificial intelligence — at least for now​

By MATT O’BRIEN

Updated 9:31 AM EDT, July 30, 2024

The White House is coming out in favor of “open-source” artificial intelligence technology, arguing in a report Tuesday that there’s no need right now for restrictions on companies making key components of their powerful AI systems widely available.

“We recognize the importance of open systems,” said Alan Davidson, an assistant secretary of the U.S. Commerce Department, in an interview with The Associated Press.

As part of a sweeping executive order on AI last year, President Joe Biden gave the U.S. Commerce Department until July to talk to experts and come back with recommendations on how to manage the potential benefits and risks of so-called open models.

The term “open-source” comes from a decades-old practice of building software in which the code is free or widely accessible for anyone to examine, modify and build upon.

However, open-source AI development involves more than just code and computer scientists differ on how to define it depending on which components of the technology are publicly available, and if there are restrictions limiting its use.

The report is the U.S. government’s first to delve into a tech industry debate between developers such as ChatGPT-maker OpenAI advocating closing off their models’ inner workings to guard against misuse, and others, such as Meta Platforms CEO Mark Zuckerberg, who have lobbied for a more open approach they say favors innovation.

“A year ago, there was a strong narrative about risk and long-term concerns about AI systems being too powerful,” said Davidson, administrator of the National Telecommunications and Information Administration. “We continue to have concerns about AI safety, but this report reflects a more balanced view that shows that there are real benefits in the openness of these technologies.”

The NTIA’s report says “current evidence is not sufficient” to warrant restrictions on AI models with “widely available weights.” Weights are numerical values that influence how an AI model performs. But it also says U.S. officials must continue to monitor potential dangers and “take steps to ensure that the government is prepared to act if heightened risks emerge.”

Though set in motion last fall, Tuesday’s report comes at a time when AI policies are now a subject of U.S. election politics in the presidential race between Vice President Kamala Harris and former President Donald Trump.

Trump’s running mate, Sen. JD Vance, has previously voiced strong support for open-source AI, warning that CEOs of big technology companies are pushing for regulations that could entrench their incumbent positions.

 

[bnew]

 

[bnew]

 

[bnew]

 

[bnew]

 

[bnew]

Spoiler: full text

1/1
GPT-4o mini launched 4 days ago.

already processing more than 200B tokens per day!

very happy to hear how much people are liking the new model.


To post tweets in this format, more info here: https://www.thecoli.com/threads/tips-and-tricks-for-posting-the-coli-megathread.984734/post-52211196

 

[bnew]

Spoiler: full text

1/1
Customize GPT-4o mini for your application with fine-tuning. Available today to tier 4 and 5 users, we plan to gradually expand access to all tiers. First 2M training tokens a day are free, through Sept 23.

https://platform.openai.com/docs/guides/fine-tuning


To post tweets in this format, more info here: https://www.thecoli.com/threads/tips-and-tricks-for-posting-the-coli-megathread.984734/post-52211196

 

[bnew]

Spoiler: thread continued

Spoiler: full text

1/12
At @a16z, we're big believers in the future of AI art.

We couldn't be more excited to share a recap of our first artist retreat hosted last month in NYC.

So grateful to all of the creatives and companies who spent time with us. Hear from them

2/12
@nickfloats on the future of art in the age of AI.

"We're definitely at a turning point in art history.

There's now a collaboration between man and machine in a way that's much deeper than it's been in the past."

3/12
@KingWillonius on his creative process with AI tools.

"Sometimes it might take 20 times regenerating a song or an image.

But that's the fun part of the process - a lot of times, you stumble upon something brilliant on accident. When that happens, it feels magical."

4/12
@RalFingerLP on how the power of the AI art community.

"When I upload a model, the creative way people utilize my stuff is so exciting.

I constantly refresh the site [@HelloCivitai] to see if anyone uploaded an image."

5/12
And finally, some takes on the AI art debate.

"It's the means to get somewhere, it's not the final result....there's always going to be naysayers, but that's fine—we're going to be ahead of them by a mile"
-Roustam Mirzoev

(more from @maxescu, @DaveJWVillalva, @GerdeGotIt)

6/12
For more on this event and our investments in AI creative tools, check out:

Elevating AI + Art | Andreessen Horowitz

7/12
None of the above should be taken as investment advice or an advertisement for investment services.

Some of the companies mentioned are portfolio companies of a16z.

A list of investments made by a16z is available at https://a16z.com/portfolio.

8/12
Oh also forgot to mention!

We'll be posting more content from all the companies that participated over the next week or so - keep an eye out.

In the meantime, thanks to @elevenlabsio, @krea_ai, @HelloCivitai, @ViggleAI, @ideogram_ai, @udiomusic

9/12
when's the next one?

10/12
haha we need to start planning ASAP!

11/12
Such a cool initiative! Gives me Bell Labs vibes

12/12



To post tweets in this format, more info here: https://www.thecoli.com/threads/tips-and-tricks-for-posting-the-coli-megathread.984734/post-52211196

 

[bnew]

 

[bnew]

Silicon Valley shaken as open-source AI models Llama 3.1 and Mistral Large 2 match industry leaders

Meta's Llama 3.1 and Mistral Large 2 open-source AI models challenge industry leaders, reshaping the AI landscape and democratizing access to cutting-edge technology.

venturebeat.com

Silicon Valley shaken as open-source AI models Llama 3.1 and Mistral Large 2 match industry leaders​

Michael Nuñez@MichaelFNunez

July 26, 2024 6:18 PM

Credit: VentureBeat made with Midjourney

---

Open-source artificial intelligence has reached a watershed moment, challenging the long-held dominance of proprietary systems and promising to reshape the AI landscape.

This week, two significant developments have propelled open-source AI models to the forefront of technological capability, potentially democratizing access to cutting-edge AI tools.

On Tuesday, Mark Zuckerberg, chief executive of Meta, unveiled Llama 3.1, declaring it had achieved “frontier-level” status.

This bold claim suggests that Meta’s freely available AI now rivals the most advanced systems from industry leaders like OpenAI, Google, and Anthropic.

Just a day later, Mistral, an emerging French AI lab, released Mistral Large 2, a model that reportedly matches or surpasses existing top-tier systems, particularly in multilingual applications.

In my experience, the accuracy gap between open-source and proprietary is negligible now and open-source is cheaper, faster, more customizable & sustainable for companies!

No excuse anymore not to become an AI builder based on open-source AI (vs outsourcing to APIs)! pic.twitter.com/3bW5aA1tNt

— clem ? (@ClementDelangue) July 24, 2024

These back-to-back releases mark a pivotal shift in the AI world. For years, tech giants have jealously guarded their most powerful AI models, citing concerns over safety, potential misuse, and competitive advantage.

This week’s developments have shattered that paradigm, igniting debates about equity, innovation, and the ethical implications of democratizing such transformative technology.

Industry experts are hailing this week’s developments as a potential turning point in AI history, comparable to pivotal moments that have sparked technological revolutions in the past.

And a second GPT-4 class open model just got released. x.com pic.twitter.com/QmWvF6r0uE

— Ethan Mollick (@emollick) July 24, 2024

The sudden availability of frontier-level open-source models is expected to dramatically accelerate AI development globally, potentially reshaping entire industries and altering the balance of power in the tech world.

This rapid democratization of cutting-edge AI capabilities could usher in a new era of innovation and competition, with far-reaching consequences for businesses, researchers, and society at large.

This might be the biggest moment for Open-Source AI.

Meta just released Llama 3.1 and a 405 billion parameter model, the most sophisticated open model ever released.

It already outperforms GPT-4o on several benchmarks. pic.twitter.com/loeBdGlJLk

— Lior (@AlphaSignalAI) July 23, 2024

​

Open-source challengers shake up the AI status quo​

The implications of this week’s announcements are far-reaching. Smaller companies and individual developers can now access sophisticated AI capabilities without the hefty price tags or vendor lock-in associated with proprietary systems. This democratization could fuel an unprecedented wave of innovation, as diverse minds from around the globe contribute to and build upon these powerful tools.

However, the widespread availability of advanced AI also raises new challenges. Organizations must now grapple with how to differentiate themselves in a world where cutting-edge AI capabilities are becoming commoditized. The onus falls on business leaders and technical decision-makers to rapidly develop strategies that leverage these open technologies while adding unique value.

The geopolitical ramifications of this shift are equally significant. As AI becomes increasingly central to national competitiveness, the proliferation of open-source models could alter the global balance of power in technology. Countries and regions that effectively harness these openly available resources may gain significant advantages in AI development and application.

​

A double-edged sword: The thrilling and terrifying dawn of AI for all​

Despite the excitement, skeptics urge caution in accepting claims of parity with top proprietary models at face value.

The AI field is known for its rapid advancements and shifting benchmarks, making “frontier-level” a moving target. Moreover, raw model capability is just one factor in AI system effectiveness; data quality, fine-tuning, and application-specific optimizations play crucial roles in real-world performance.

The abrupt open-sourcing of frontier-level AI also intensifies ongoing debates about AI safety and ethics. While transparency can aid in identifying and addressing biases or vulnerabilities, it may also lower barriers for malicious actors seeking to exploit these powerful tools. The AI community now faces the urgent challenge of striking a delicate balance between openness and responsible development.

For policymakers, this week’s developments underscore the critical need for adaptive regulatory frameworks that can keep pace with technological advancements while ensuring public safety and ethical use of AI. The tech industry may need to rapidly reevaluate business models and competitive strategies in a landscape where cutting-edge AI capabilities have suddenly become widely accessible.

​

Navigating the new frontier: Collaboration, ethics, and the future of AI​

As the dust settles on this landmark week, the true impact of these milestones will be determined by how effectively the global community harnesses the potential of open-source AI while mitigating its risks.

The sudden democratization of frontier-level AI has the potential to accelerate innovation, reshape industries, and fundamentally alter our relationship with artificial intelligence.

In this new era, collaboration and ethical considerations will be paramount. The open-source AI revolution promises to unlock unprecedented possibilities, but it also demands a heightened sense of responsibility from developers, businesses, and society as a whole.

As we navigate this transformative period, one thing is clear: the future of AI is becoming more open, more accessible, and more participatory than ever before, and the pace of change is accelerating rapidly.

 

[bnew]

 

[bnew]

Google brings Gemini-powered search history and Lens to Chrome desktop | TechCrunch

Google Thursday said that it is introducing new Gemini-powered features for Chrome's desktop version, including Lens for desktop, tab compare for shopping

techcrunch.com

Google brings Gemini-powered search history and Lens to Chrome desktop​

Ivan Mehta

9:00 AM PDT • August 1, 2024

Comment

Image Credits: Google

Google Thursday said that it is introducing new Gemini-powered features for Chrome’s desktop version, including Lens for desktop, tab compare for shopping assistance, and natural language integration for search history.

Years after introducing and evolving Google Lens on mobile, the feature is finally coming to desktop. Rolling out to users across the world in the coming days, Lens will live in the address bar, as well as the three-dot menu. After clicking, you can select a part of a page and ask more questions to get search results.



You can also tap on objects, such as someone’s backpack in a picture, and ask questions through multi-search to find a similar item in different colors or brands. Depending on the question you ask, you might also get AI Overviews in answers.

In addition to searching for shoppable items, users can also find out how much sunlight a plant needs, for example, or get help understanding a math equation.

Image Credits:Google

Google is also introducing a new feature called Tab Compare to aid shopping. In the coming weeks, Chrome will offer an AI-powered summary of similar items you might be searching across different tabs. For instance, if you are searching for a new Bluetooth speaker, the feature will show details such as product specs, features, price and ratings in one place, even when you’re looking at these details across different pages.

Image Credits:Google

One of the most useful updates of this lot is the ability to search your browsing history through natural language queries. Sometimes you don’t remember what page you visited apart from a few details. The company is rolling out AI-powered history search in the coming weeks as an opt-in feature for U.S. users.

Image Credits:Google

An example of a natural language query is, “What was that ice cream shop I looked at last week?” Google uses a combination of URL, title, and contents of the page to show search results.

The company said that it doesn’t use this data to train Gemini and won’t surface any information from the incognito session. Google currently can’t process AI-powered search history locally, however, so it uses cloud capacity to return results.

In January, the company introduced AI-powered features such as a writing assistant, tab organizer, and theme creator. In May, it rolled out a way to mention Gemini and ask the chatbot questionsdirectly from the address bar.

 

[bnew]

OpenAI releases ChatGPT's hyperrealistic voice to some paying users | TechCrunch

OpenAI began rolling out ChatGPT's Advanced Voice Mode on Tuesday, giving users their first access to GPT-4o's hyperrealistic audio responses. The alpha

techcrunch.com

OpenAI releases ChatGPT’s hyperrealistic voice to some paying users​

Maxwell Zeff

11:30 AM PDT • July 30, 2024

Comment

Image Credits: OpenAI

OpenAI began rolling out ChatGPT’s Advanced Voice Mode on Tuesday, giving users their first access to GPT-4o’s hyperrealistic audio responses. The alpha version will be available to a small group of ChatGPT Plus users today, and OpenAI says the feature will gradually roll out to all Plus users in the fall of 2024.

When OpenAI first showcased GPT-4o’s voice in May, the feature shocked audiences with quick responses and an uncanny resemblance to a real human’s voice – one in particular. The voice, Sky, resembled that of Scarlett Johansson, the actress behind the artificial assistant in the movie “Her.” Soon after OpenAI’s demo, Johansson said she refused multiple inquiries from CEO Sam Altman to use her voice, and after seeing GPT-4o’s demo, hired legal counsel to defend her likeness. OpenAI denied using Johansson’s voice, but later removed the voice shown in its demo. In June, OpenAI said it would delay the release of Advanced Voice Mode to improve its safety measures.

One month later, and the wait is over (sort of). OpenAI says the video and screensharing capabilities showcased during its Spring Update will not be part of this alpha, launching at a “later date.” For now, the GPT-4o demo that blew everyone away is still just a demo, but some premium users will now have access to ChatGPT’s voice feature shown there.

ChatGPT can now talk and listen​

You may have already tried out the Voice Mode currently available in ChatGPT, but OpenAI says Advanced Voice Mode is different. ChatGPT’s old solution to audio used three separate models: one to convert your voice to text, GPT-4 to process your prompt, and then a third to convert ChatGPT’s text into voice. But GPT-4o is multimodal, capable of processing these tasks without the help of auxiliary models, creating significantly lower latency conversations. OpenAI also claims GPT-4o can sense emotional intonations in your voice, including sadness, excitement or singing.

In this pilot, ChatGPT Plus users will get to see first hand how hyperrealistic OpenAI’s Advanced Voice Mode really is. TechCrunch was unable to test the feature before publishing this article, but we will review it when we get access.

OpenAI says it’s releasing ChatGPT’s new voice gradually to closely monitor its usage. People in the alpha group will get an alert in the ChatGPT app, followed by an email with instructions on how to use it.

In the months since OpenAI’s demo, the company says it tested GPT-4o’s voice capabilities with more than 100 external red teamers who speak 45 different languages. OpenAI says a report on these safety efforts is coming in early August.

The company says Advanced Voice Mode will be limited to ChatGPT’s four preset voices – Juniper, Breeze, Cove and Ember – made in collaboration with paid voice actors. The Sky voice shown in OpenAI’s May demo is no longer available in ChatGPT. OpenAI spokesperson Lindsay McCallum says “ChatGPT cannot impersonate other people’s voices, both individuals and public figures, and will block outputs that differ from one of these preset voices.”

OpenAI is trying to avoid deepfake controversies. In January, AI startup ElevenLabs’s voice cloning technology was used to impersonate President Biden, deceiving primary voters in New Hampshire.

OpenAI also says it introduced new filters to block certain requests to generate music or other copyrighted audio. In the last year, AI companies have landed themselves in legal trouble for copyright infringement, and audio models like GPT-4o unleash a whole new category of companies that can file a complaint. Particularly, record labels, who have a history for being litigious, and have already sued AI song-generators Suno and Udio.

 

[bnew]

[2407.09450] Human-like Episodic Memory for Infinite Context LLMs

[Submitted on 12 Jul 2024]

Human-like Episodic Memory for Infinite Context LLMs​

Zafeirios Fountas, Martin A Benfeghoul, Adnan Oomerjee, Fenia Christopoulou, Gerasimos Lampouras, Haitham Bou-Ammar, Jun Wang

Large language models (LLMs) have shown remarkable capabilities, but still struggle with processing extensive contexts, limiting their ability to maintain coherence and accuracy over long sequences. In contrast, the human brain excels at organising and retrieving episodic experiences across vast temporal scales, spanning a lifetime. In this work, we introduce EM-LLM, a novel approach that integrates key aspects of human episodic memory and event cognition into LLMs, enabling them to effectively handle practically infinite context lengths while maintaining computational efficiency. EM-LLM organises sequences of tokens into coherent episodic events using a combination of Bayesian surprise and graph-theoretic boundary refinement in an on-line fashion. When needed, these events are retrieved through a two-stage memory process, combining similarity-based and temporally contiguous retrieval for efficient and human-like access to relevant information. Experiments on the LongBench dataset demonstrate EM-LLM's superior performance, outperforming the state-of-the-art InfLLM model with an overall relative improvement of 4.3% across various tasks, including a 33% improvement on the PassageRetrieval task. Furthermore, our analysis reveals strong correlations between EM-LLM's event segmentation and human-perceived events, suggesting a bridge between this artificial system and its biological counterpart. This work not only advances LLM capabilities in processing extended contexts but also provides a computational framework for exploring human memory mechanisms, opening new avenues for interdisciplinary research in AI and cognitive science.

Subjects:	Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG); Neurons and Cognition (q-bio.NC)
Cite as:	arXiv:2407.09450 [cs.AI]
	(or arXiv:2407.09450v1 [cs.AI] for this version)
	[2407.09450] Human-like Episodic Memory for Infinite Context LLMs

Submission history​

From: Zafeirios Fountas PhD [view email]

[v1] Fri, 12 Jul 2024 17:34:03 UTC (313 KB)

https://arxiv.org/pdf/2407.09450

---

A.I generated summary:

### Human-like Episodic Memory for Infinite Context LLMs

Imagine you have a super-smart computer program that can understand and remember lots of information, just like humans do. This program is called a Large Language Model (LLM). However, these LLMs have a big problem: they struggle to remember and use lots of information over long periods of time. This makes it hard for them to stay coherent and accurate when dealing with long sequences of data.

In contrast, humans are great at organizing and recalling personal experiences across their entire lives. Our brains break down continuous experiences into discrete events, which are stored in long-term memory. When we need to remember something, our brains use these event boundaries as access points.

This research introduces a new approach called EM-LLM (Episodic Memory Large Language Model), which integrates key aspects of human episodic memory into LLMs. EM-LLM helps these models handle practically infinite context lengths while staying computationally efficient.

#### How EM-LLM Works

1. **Event Segmentation**: EM-LLM breaks down sequences of tokens (pieces of text) into coherent episodic events using two main steps:
- **Surprise-Based Segmentation**: It identifies important moments where the model is "surprised" by new information.
- **Graph-Theoretic Boundary Refinement**: It refines these boundaries to ensure that each event is cohesive and distinct from others.

2. **Memory Retrieval**: When needed, EM-LLM retrieves these events through a two-stage process:
- **Similarity-Based Retrieval**: It selects relevant events based on how similar they are to the current query.
- **Temporal Contiguity Buffer**: It also considers the temporal order of events to maintain context.

#### Experiments

The researchers tested EM-LLM on various tasks using the LongBench dataset and compared its performance with another state-of-the-art model called InfLLM. The results showed that EM-LLM outperformed InfLLM by an overall relative improvement of 4.3%, with significant improvements in specific tasks like PassageRetrieval (33% improvement).

#### Key Findings

- The segmentation method used in EM-LLM correlates strongly with how humans perceive events.
- The combination of surprise-based segmentation and boundary refinement significantly enhances performance on long-context tasks.
- The use of both similarity-based and temporally contiguous retrieval mechanisms mimics human memory retrieval patterns.

#### Future Directions

This work opens up new avenues for interdisciplinary research between AI and cognitive science:
- Exploring more sophisticated segmentation algorithms for better performance.
- Extending this approach to enable imagination and future thinking in LLMs.
- Integrating modality-specific buffers for multi-modal tasks.

In summary, EM-LLM represents a significant advancement in language models by integrating human-like episodic memory mechanisms, enabling them to handle extended contexts efficiently and effectively.

 

[bnew]

###
I voted False, and would have done so even if you'd said the end of 2025.

For sure stock prices of Big Tech firms will go up and down, and media will alternate btw positive and negative ... that is how these dynamical systems operate....

However, I don't think the GenAI bubble will ever burst, because I think the real super-high value applications of this tech will happen fast enough to avoid CEOs of Big Tech firms burning out on GenAI ...

I don't think this will be a matter of chatbot subscriptions catching on mega big time (though smartphone assistants will get smarter fast, and chatbots will get more and more functional etc.) ... it will be more centrally about back-end integration of GenAI into all sorts of other useful software and hardware applications.

This integration is happening now, all over the place. It's having hiccups, and is working better in some cases than others, etc. It is being a learning experience, and is requiring AI devs and application designers to figure out how to work around the very real and glaring shortcomings of LLMs. But it's happening nonetheless.

Separately from my AGI R&D, I'm involved in a host of AI application projects, including for instance 2 that are LLM-based but have not yet created launchable products due to RAG not working as envisioned/advertised. HOWEVER, we seem to be getting much better results using custom variants of GraphRAG combined with clever prompt engineering... even without (yet) introducing full-on neural-symbolic approaches using OpenCog Hyperon (which we will do in these projects next year).

I think similar stories are happening all around. LLMs are proving to not be magic and not be AGI, but they are nonetheless unprecedentedly powerful AI tools, and I believe the current influx of investment into them will be enough to get them over the line in many many different verticals in terms of building really really useful applications that make loads of $$ and improve loads of lives.

(Not to overlook the damage that is done by the exploitative way data has been gathered for training many large models, though, nor the damage that is done to the Ai field by so much talent being sucked into LLMs instead of other R&D areas, etc. These negative factors totally exist and we must counter them emphatically, but they don't obviate the positives.)

It took a while for the world to figure out how to integrate ML broadly into software and hardware applications. The learning curve is going to be significantly shorter for LLMs because as we get near Singularity, so many things are going faster !! ...

What will be really interesting though is, just as industry starts hitting its stride using LLMs on the back end of various amazing applications, we're going to come out with huge research breakthroughs on putting LLMs and other DNNs together with symbolic probabilistic reasoning and creative evolutionary learning ... in the OpenCog Hyperon project, running on ASI Alliance (inc. SingularityNET, HyperCycle, NuNet) back end ..... And the process of hype, confusion and integration will begin all over again -- but EVEN FASTER, I predict, for the neural-symbolic-evolutionary early-stage-AGI wave than what we'll see with LLMs...

The human world remains confusing, self-harming and needlessly demented in many ways... but we are getting a lot of thing right nonetheless ... and the Singularity is Near ...
###