The A.I Megathread (LLM , GPT , Development)

[bnew]

Spoiler: thread continued

Spoiler: full text

1/8
SuperCoder 2.0 achieves 33.66% success rate in SWE-Bench Lite, ranking #3 globally and #1 among all Open Source Coding Systems.

Read our Technical Report on SuperCoder's results here: SuperCoder 2.0 achieves 33.66% success rate in SWE-bench Lite, ranking #3 globally & #1 among all open-source coding systems - SuperAGI

2/8
SuperCoder 2.0 integrates GPT-4o and Sonnet-3.5 to refine our dual approach of Code Search and Code Generation. By identifying precise bug locations and generating robust code patches, the system streamlines the debugging process in large codebases.

3/8
Our approach utilizes an innovative method of extracting detailed metadata and employing vector databases for advanced search capabilities. This allows us to locate specific code issues accurately and deploy solutions effectively, ensuring higher reliability in patch application.

4/8
We address Python's indentation sensitivity by regenerating entire method bodies, allowing for seamless integration of new or revised code without introducing syntax errors, which is crucial for maintaining code functionality after patches.

5/8
Try SuperCoder now: SuperAGI - Open-source Autonomous Software Systems - SuperAGI

6/8
Impressive results! I'll take a look at the technical report to learn more about SuperCoder 2.0's success in SWE-Bench Lite. Keep up the great work!

lets connect

7/8
An impressive achievement for SuperCoder 2.0!

8/8
Hello, do you like to promote your account? We have the ability to promote your account and increase your followers. We help you create a strong and effective account with a very attractive price. Leave me a private message about the prices.
/search?q=#HisMan3 /search?q=#Schoof /search?q=#ssmonth24
/search?q=#Bitcoin


To post tweets in this format, more info here: https://www.thecoli.com/threads/tips-and-tricks-for-posting-the-coli-megathread.984734/post-52211196

Spoiler: larger images

 

[bnew]

With Google in its sights, OpenAI unveils SearchGPT | TechCrunch

OpenAI is previewing an AI-powered search experience called SearchGPT, some aspects of which the company says could make their way into ChatGPT.

techcrunch.com

With Google in its sights, OpenAI unveils SearchGPT​

Kyle Wiggers

11:52 AM PDT • July 25, 2024

Comment

Image Credits: Bryce Durbin / TechCrunch

OpenAI may have designs to get into the search game — challenging not only upstarts like Perplexity, but Google and Bing, too.

The company on Thursday unveiled SearchGPT, a search feature designed to give “timely answers” to questions, drawing from web sources.

UI-wise, SearchGPT isn’t too far off from OpenAI’s chatbot platform ChatGPT. You type in a query, and SearchGPT serves up information and photos from the web along with links to relevant sources, at which point you can ask follow-up questions or explore additional, related searches in a sidebar.

Some searches take into account your location. In a support doc, OpenAI writes that SearchGPT “collects and shares” general location information with third-party search providers to improve the accuracy of results (e.g. showing a list of restaurants nearby or the weather forecast). SearchGPT also allows users to share more precise location information via a toggle in the settings menu.

Powered by OpenAI models (specifically GPT-3.5, GPT-4 and GPT-4o), SearchGPT — which OpenAI describes as a prototype — is launching today for “a small group” of users and publishers. (There’s a waitlist here.) OpenAI says that it plans to integrate some features of SearchGPT into ChatGPT in the future.

“Getting answers on the web can take a lot of effort, often requiring multiple attempts to get relevant results,” OpenAI writes in a blog post. “We believe that by enhancing the conversational capabilities of our models with real-time information from the web, finding what you’re looking for can be faster and easier.”

Image Credits:SearchGPT

It’s long been rumored that OpenAI’s taken an interest in launching a Google killer of sorts. The Information reported in February that a product — or at least a pilot — was in the works. But the launch of SearchGPT comes at an inopportune moment: as AI-powered search tools are under justifiable fire for plagiarism, inaccuracies and content cannibalism.

Google’s take on AI-powered search, AI Overviews, infamously suggested putting glue on a pizza. The Browser Company’s Arc Search told one reporter that cut-off toes will grow back. AI search engine Genspark at one time readily recommended weapons that could be used to kill someone. And Perplexity ripped off news articles written by other outlets, including CNBC, Bloomberg, and Forbes, without giving credit or attribution.

AI-generated overviews threaten to cannibalize traffic to the sites from which they source their info. Indeed, they already are, with one study finding that AI Overviews could negatively affect about 25% of publisher traffic due to the de-emphasis on article links.

OpenAI — taking a page out of Perplexity’s ongoing apology tour — is positioning SearchGPT as a more responsible, measured deployment.

OpenAI says that SearchGPT “prominently cites and links” to publishers in searches with “clear, in-line, named attribution.” It also says that it’s working with publishers to design the experience and providing a way for website owners to manage how their content appears in search results.

“Importantly, SearchGPT is about search and is separate from training OpenAI’s generative AI foundation models. Sites can be surfaced in search results even if they opt out of generative AI training,” OpenAI clarifies in the blog post. “We are committed to a thriving ecosystem of publishers and creators.”

It’s a bit tough to take a company that once scraped millions of YouTube transcripts without permission to train its models at their word. But we’ll see how the SearchGPT saga unfolds.

 

[bnew]

A new Chinese video-generating model appears to be censoring politically sensitive topics | TechCrunch

A new video-generating AI model, Kling, won't readily generate videos of topics deemed politically sensitive.

techcrunch.com

A new Chinese video-generating model appears to be censoring politically sensitive topics​

Kyle Wiggers

4:07 PM PDT • July 24, 2024

Comment

Image Credits: Photo by VCG/VCG via Getty Images

A powerful new video-generating AI model became widely available today — but there’s a catch: The model appears to be censoring topics deemed too politically sensitive by the government in its country of origin, China.

The model, Kling, developed by Beijing-based company Kuaishou, launched in waitlisted access earlier in the year for users with a Chinese phone number. Today, it rolled out for anyone willing to provide their email. After signing up, users can enter prompts to have the model generate five-second videos of what they’ve described.

Kling works pretty much as advertised. Its 720p videos, which take a minute or two to generate, don’t deviate too far from the prompts. And Kling appears to simulate physics, like the rustling of leaves and flowing water, about as well as video-generating models like AI startup Runway’s Gen-3 and OpenAI’s Sora.

But Kling outright won’t generate clips about certain subjects. Prompts like “Democracy in China,” “Chinese President Xi Jinping walking down the street” and “Tiananmen Square protests” yield a nonspecific error message.

Image Credits:Kuaishou

The filtering appears to be happening only at the prompt level. Kling supports animating still images, and it’ll uncomplainingly generate a video of a portrait of Jinping, for example, as long as the accompanying prompt doesn’t mention Jinping by name (e.g., “This man giving a speech”).

We’ve reached out to Kuaishou for comment.

Image Credits:Kuaishou

Kling’s curious behavior is likely the result of intense political pressure from the Chinese government on generative AI projects in the region.

Earlier this month, the Financial Times reported that AI models in China will be tested by China’s leading internet regulator, the Cyberspace Administration of China (CAC), to ensure that their responses on sensitive topics “embody core socialist values.” Models are to be benchmarked by CAC officials for their responses to a variety of queries, per the Financial Times report — many related to Jinping and criticism of the Communist Party.

Reportedly, the CAC has gone so far as to propose a blacklist of sources that can’t be used to train AI models. Companies submitting models for review must prepare tens of thousands of questions designed to test whether the models produce “safe” answers.

The result is AI systems that decline to respond on topics that might raise the ire of Chinese regulators. Last year, the BBC found that Ernie, Chinese company Baidu’s flagship AI chatbot model, demurred and deflected when asked questions that might be perceived as politically controversial, like “Is Xinjiang a good place?” or “Is Tibet a good place?”

The draconian policies threaten to slow China’s AI advances. Not only do they require scouring data to remove politically sensitive info, but they also necessitate investing an enormous amount of dev time in creating ideological guardrails — guardrails that might still fail, as Kling exemplifies.

From a user perspective, China’s AI regulations are already leading to two classes of models: some hamstrung by intensive filtering and others decidedly less so. Is that really a good thing for the broader AI ecosystem?

 

[bnew]

Google makes its Gemini chatbot faster and more widely available | TechCrunch

Google is upgrading its Gemini chatbot experience with a newer model, Gemini 1.5 Flash, and bringing Gemini to more places.

techcrunch.com

Google makes its Gemini chatbot faster and more widely available​

Kyle Wiggers

9:00 AM PDT • July 25, 2024

Comment

Image Credits: Lorenzo Di Cola/NurPhoto / Getty Images

In its bid to maintain pace with generative AI rivals like Anthropic and OpenAI, Google is rolling out updates to the no-fee tier of Gemini, its AI-powered chatbot. The updates are focused on making the platform more performant — and more widely available.

Starting Thursday, Gemini 1.5 Flash — a lightweight multimodal model Google announced in May — will be available on the web and mobile in 40 languages and around 230 countries. Google claims that Gemini 1.5 Flash delivers upgrades in quality and latency, with especially noticeable improvements in reasoning and image understanding.

In a boon for Google, it might also be cheaper to run on the back end.

At Gemini 1.5 Flash’s unveiling, Google emphasized that the model was a “distilled” and highly efficient version of Gemini 1.5 Pro, built for what the company described as “narrow,” “high-frequency” generative AI workloads. Given the overhead of serving a chatbot platform such as Gemini (see: OpenAI’s ChatGPT bills), Google’s no doubt eager to jump on cost-reducing opportunities, particularly if those opportunities have the fortunate side effect of boosting performance in other areas.

Beyond the new base model, Google says that it’s expanding Gemini’s context window to 32,000 tokens, which amounts to roughly 24,000 words (or 48 pages of text).

Image Credits:Google

Context, or context window, refers to the input data (e.g., text) that a model considers before generating output (e.g., additional text). A few of the advantages of models with larger contexts are that they can summarize and reason over longer text snippets and files (at least in theory), and that — in a chatbot context — they’re less likely to forget topics that were recently discussed.

The ability to upload files to Gemini for analysis previously required Gemini Advanced, the paid edition of Gemini gated behind Google’s $20-per-month Google One AI Premium Plan. But Google says that it’ll soon enable file uploads from Google Drive and local devices for all Gemini users.

“You’ll be able to do things like upload your economics study guide and ask Gemini to create practice questions,” Amar Subramanya, VP of engineering at Google, wrote in a blog post shared with TechCrunch. “Gemini will also soon be able to analyze data files for you, allowing you to uncover insights and visualize them through charts and graphics.”

To attempt to combat hallucinations — instances where a generative AI model like Gemini 1.5 Flash makes things up — Google is previewing a feature that displays links to related web content beneath certain Gemini-generated answers. English-language Gemini users in select territories will see a “chip” icon at the end of a Gemini-generated paragraph with a link to websites — or emails, if you’ve given Gemini permission to access your Gmail inbox — where you can dive deeper.

The move comes after revelations that Google’s generative AI models are prone to hallucinating quite badly — for example, suggesting nontoxic glue in a pizza recipe and inventing fake book reviews attributed to real people. Google earlier this year released a “double check” feature in Gemini designed to highlight Gemini-originated statements that other online sources corroborate or contradict. But the related content links appear to be an effort to make more transparent which sources of info Gemini might be drawing from.

The question in this reporter’s mind is how often and accurately Gemini will surface related links. TBD.

Google’s not waiting to flood the channels, though.

After debuting Gemini in Messages for select devices earlier in the year, Google is rolling the feature in the European Economic Area (EEA), U.K. and Switzerland, with the ability to chat in newly added languages such as French, Polish and Spanish. Users can pull up Gemini in Messages by tapping the “Start chat” button and selecting Gemini as a chat partner.

Google’s also launching the Gemini mobile app in more countries, and expanding Gemini access to teenagers globally.

The company introduced a teen-focused Gemini experience in June, allowing students to sign up using their school accounts — though not in all countries. In the coming week, that’ll change as Gemini becomes available to teens in every country and region that Gemini is normally available to adults.

Coinciding with the rollout, Google says that it’s putting “additional policies and safeguards” in place to protect teens — without going into detail. A new teen-tailored onboarding process is also in tow, along with an “AI literacy guide” to — as Google phrases it — “help teens use AI responsibly.”

It’s the subject of great debate whether kids are leveraging generative AI tools in the ways they were intended, or abusing them. Google is surely eager to avoid headlines suggesting Gemini is a plagiaristic essay generator or capable of giving teens poorly conceived advice on personal problems, and thus taking what steps it can to prevent the worst from occurring.

 

[bnew]

Adobe releases new Firefly AI tools for Illustrator and Photoshop | TechCrunch

Adobe released new Firefly tools for Photoshop and Illustrator on Tuesday, offering graphic designers more ways to use the company's in-house AI models. Adobe is doubling down on Firefly by releasing new AI tools for Illustrator and Photoshop.

techcrunch.com

Adobe releases new Firefly AI tools for Illustrator and Photoshop​

Maxwell Zeff

6:00 AM PDT • July 23, 2024

Comment

Image Credits: Adobe

Adobe released new Firefly tools for Photoshop and Illustrator on Tuesday, offering graphic designers more ways to use the company’s in-house AI models. Adobe’s new features let creative workers describe what they want with brief prompts and receive AI-generated textures or images that could otherwise take hours to create.

As Adobe doubles down on AI, the company walks a fine line with some of its loyal users who feel threatened by it. The company trained Firefly on the work of many creative workers and even pays Adobe Stock’s photographers and illustrators an annual bonus for this privilege. In light of this, Adobe says it is taking a “creator-friendly approach,” offering Creative Cloud customers a limited number of generative credits every month at no added cost.

“This is all value that’s baked into their Creative Cloud plan,” Adobe’s VP of product marketing, Deepa Subramaniam, said in an interview with TechCrunch. “We just want to put this technology in the hands of our users, so they’re very generous credit limits.”

Illustrator is releasing a new tool in beta, “Generative Shape Fill,” where users can add details and textures to shapes through text prompts or by selecting a style reference. The feature is powered by an updated beta version of Adobe’s Firefly Vector model, which is also being released on Tuesday. In a demo, Adobe Creative Cloud Evangelist Paul Trani showed how the new feature comes with sliders to select how much detail a creator can add. After Firefly generates those details, Illustrator allows the user to edit them; that sets it apart from most AI image generators.

For Photoshop, Adobe is making Firefly’s text-to-image generator broadly available, allowing users to create AI images in the application by pressing “Generate Image” and describing what they’d like. This is powered by Adobe Firefly’s Image 3 Foundation model and was previously only available in beta.

Even though Adobe isn’t charging a premium subscription for generative AI like some competitors, it’s still seeing returns. On its latest earnings call, the company said customers were converting to pricier plans as a means to get more generative credits for Firefly. (One generative credit is equivalent to one generation for other AI tools.) Since launch in March 2023, Adobe says Firefly has generated more than 9 billion images.

Illustrator also received an array of new features that don’t use generative AI. Adobe’s new Dimension tool allows users to calculate lengths and angles of an image in Illustrator. Illustrator’s beta tool, Mockup, allows you to realistically place a logo on any product just by uploading an image. Previously, Mockup would only let you place a logo on a predetermined set of images. With Retype, Illustrator can recognize fonts, match them, and allow you to use them elsewhere.

As for Photoshop, the pixel-based workspace is making a new Selection Brush tool and Adjustment Brush tool generally available, both aiming to streamline repetitive tasks.

 

[bnew]

Developers aren’t worried that gen AI will steal their jobs, Stack Overflow survey reveals

Developers aren’t afraid gen AI will take their jobs, but they don’t fully trust the technology yet either.

venturebeat.com

Developers aren’t worried that gen AI will steal their jobs, Stack Overflow survey reveals​

Sean Michael Kerner@TechJournalist

July 24, 2024 7:00 AM

Credit: Image generated by VentureBeat using Stable Diffusion 3

Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More

---

Generative AI is having a dramatic impact on many industries, including software development.

In a revealing snapshot of the global software development ecosystem, developer knowledge platformStack Overflow is out today with a new report that exposes a nuanced interplay between AI and the coding community.

Stack Overflow’s 2024 Developer Survey is based on responses from more than 65,000 developers across 185 countries. The survey offers key insights into the evolving tech landscape, focusing on gen AI and its impact on developers. Stack Overflow itself is no stranger to the world of gen AI. Stack Overflow has its own set of tools, includingOverflowAI to help developers.

Among the key findings in Stack Overflow’s 2024 Developer Survey:

• 
  AI tool usage among developers increased to 76% in 2024, up from 70% in 2023
• 
  Despite increased usage, AI favorability decreased from 77% to 72%
• 
  Only 43% of respondents trust the accuracy of AI tools
• 
  81% of developers cite increased productivity as the top benefit of AI tools
• 
  Misinformation emerges as the top AI-related ethical concern (79%)

A surprising finding was that 70% of professional developers don’t see AI as a threat to their jobs.

“Although we at Stack Overflow have been vocal about the fact that AI tools should help create more opportunities for developers – given the economic climate of late, I was expecting responses to be closer to 50/50,” Erin Yepis, senior analyst, market research and insights, told VentureBeat.

Why gen AI isn’t going to replace developers anytime soon​

The fear from some that gen AI might replace developers is actually somewhat misplaced. What might actually be the case is that gen AI is increasing the number of developers rather than reducing the need for them.

“We believe that generative AI will democratize coding and grow the developer community by several folds,” Ryan Polk, Chief Product Officer at Stack Overflow told VentureBeat. “If there are 20-25 million developers now, this could lead to a 10x growth of this important group.”

Polk noted that gen AI coding tools can help developers in their day-to-day efforts. He sees it as a “Better Together” approach in terms of gen AI development tools and sites like Stack Overflow. For example, Polks said that AI-code completion and generator tools paired with deep context and knowledge tools like Stack Overflow provide a powerful combination. He believes AI-powered code generators will reduce time spent on boilerplate code, letting developers focus on complex problems.

“While gen AI tools can complement the resources available to developers, only Stack Overflow’s community expertise with accurate and sourced content complements these tools and is an invaluable resource for coding teams,” Polk said.

The flow of information from Stack Overflow to gen AI works both ways. Users can come to Stack Overflow to get answers and Stack Overflow partners with leading AI vendors to help train models. Among Stack Overflow’s partnerships is one with Google Cloud which was announced in February. Another key partnership for Stack Overflow emerged in May with OpenAI.

Why developers have a less favorable view of gen AI this year​

Among the declining statistics in the 2024 report is a metric on favorability. In 2023, 77% of respondents had a favorable view of gen AI developer tools, which fell to 72% in 2024.

Yepis explained that favorability is top-box grouping for the Likert scale question of “How favorable is your stance on using AI tools as part of your development workflow?”

“Our hypothesis is that more people are getting a chance to try gen AI tools and are disappointed in their experience,” Yepis said. “Context is key, there is nuance in the type of role and the type of AI tool that could be contributing to this.”

Why developers lack trust in gen AI tools and how that (might) improve​

A lack of trust in gen AI developer tools was another top-level finding in the report. This ties into the well-known issue of AI hallucination, which is a significant concern for developers.

Polk noted that attribution and the ability to decipher context continue to be issues for users. From this year’s survey, the top three ethical issues related to AI that developers are concerned with: AI’s potential to circulate misinformation (79%), missing or incorrect attribution for sources of data (65%) and bias that does not represent a diversity of viewpoints (50%).

“Gen AI is only as good as its data, garbage in, garbage out,” he said. “The truth is key for AI models.”

In Polk’s view, Stack Overflow and its users have a role to play in helping to overcome trust issues. He noted that Stack Overflow believes engaged developers and shared answers from a community will ensure AI’s future success. Polk emphasized that user trust in data, technology, and community knowledge is crucial amid misinformation in gen AI solutions.

“This is at the heart of why Stack has been focused on our OverflowAPI partnerships with AI and cloud companies to responsibly share content from Stack Overflow for various AI and [large language model] LLM use cases,” Polk said. “Our goal with OverflowAPI, and our work to advance the era of socially responsible AI, is to set new standards with vetted, trusted, and accurate data that will be the foundation on which technology solutions are built and delivered to our users.”

 

[bnew]

Stability AI steps into a new gen AI dimension with Stable Video 4D

Stability AI is adding the dimension of time to gen AI video with new 4D model.

venturebeat.com

Stability AI steps into a new gen AI dimension with Stable Video 4D​

Sean Michael Kerner@TechJournalist

July 24, 2024 8:00 AM

Credit: Image generated by VentureBeat using Stable Diffusion 3

Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More

---

Stability AI is expanding its growing roster of generative AI models, quite literally adding a new dimension with the debut of Stable Video 4D.

While there is a growing set of gen AI tools for video generation, including OpenAI’s Sora, Runway, Haiper and Luma AI among others, Stable Video 4D is something a bit different. Stable Video 4D builds on the foundation of Stability AI’s existing Stable Video Diffusion model, which converts images into videos. The new model takes this concept further by accepting video input and generating multiple novel-view videos from 8 different perspectives.

“We see Stable Video 4D being used in movie production, gaming, AR/VR, and other use cases where there is a need to view dynamically moving 3D objects from arbitrary camera angles,” Varun Jampani, team lead, 3D Research at Stability AI told VentureBeat.

Stable Video 4D is different than just 3D for gen AI​

This isn’t Stability AI’s first foray beyond the flat world of 2D space.

In March, Stable Video 3D was announced, enabling users to generate short 3D video from an image or text prompt. Stable Video 4D is going a significant step further. While the concept of 3D, that is 3 dimensions, is commonly understood as a type of image or video with depth, 4D isn’t perhaps as universally understood.

Jampani explained that the four dimensions include width (x), height (y), depth (z) and time (t). That means Stable Video 4D is able to view a moving 3D object from various camera angles as well as at different timestamps.

“The key aspects that enabled Stable Video 4D are that we combined the strengths of our previously-released Stable Video Diffusion and Stable Video 3D models, and fine-tuned it with a carefully curated dynamic 3D object dataset,” Jampani explained.

Jampani noted that Stable Video 4D is a first-of-its-kind network where a single network does both novel view synthesis and video generation. Existing works leverage separate video generation and novel view synthesis networks for this task.

He also explained that Stable Video 4D is different from Stable Video Diffusion and Stable Video 3D, in terms of how the attention mechanisms work.

“We carefully design attention mechanisms in the diffusion network which allow generation of each video frame to attend to its neighbors at different camera views or timestamps, thus resulting in better 3D coherence and temporal smoothness in the output videos,” Jampani said.

How Stable Video 4D works differently than gen AI infill​

With gen AI tools for 2D image generation the concept of infill and outfill, to fill in gaps, is well established. The infill/outfill approach however is not how Stable Video 4D works.

Jampani explained that the approach is different from generative infill/outfill, where the networks typically complete the partially given information. That is, the output is already partially filled by the explicit transfer of information from the input image.

“Stable Video 4D completely synthesizes the 8 novel view videos from scratch by using the original input video as guidance,” he said. “There is no explicit transfer of pixel information from input to output, all of this information transfer is done implicitly by the network.”

Stable Video 4D is currently available for research evaluation on Hugging Face. Stability AI has not yet announced what commercial options will be available for it in the future.

“Stable Video 4D can already process single-object videos of several seconds with a plain background,” Jampani said. “We plan to generalize it to longer videos and also to more complex scenes.”

 

[bnew]

Mistral shocks with new open model Mistral Large 2, taking on Llama 3.1

On the Multilingual MMLU tasks, Mistral Large 2 performed on par with Meta’s Llama 3.1-405B while delivering significant cost benefits.

venturebeat.com

Mistral shocks with new open model Mistral Large 2, taking on Llama 3.1​

Shubham Sharma@mr_bumss

July 24, 2024 11:21 AM

Credit: VentureBeat made with Midjourney V6

Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More

---

The AI race is picking up pace like never before. Following Meta’s move just yesterday to launch its new open source Llama 3.1 as a highly competitive alternative to leading closed-source “frontier” models, French AI startup Mistral has also thrown its hat in the ring.

The startup announced the next generation of its flagship open source model with 123 billion parameters: Mistral Large 2. However, in an important caveat, the model is only licensed as “open” for non-commercial research uses, including open weights, allowing third-parties to fine-tune it to their liking.

For those seeking to use it for commercial/enterprise-grade applications, they will need to obtain a separate license and usage agreement from Mistral, as the company states in its blog post and in an X post from research scientist Devendra Singh Chaplot.

Super excited to announce Mistral Large 2
– 123B params – fits on a single H100 node
– Natively Multilingual
– Strong code & reasoning
– SOTA function calling
– Open-weights for non-commercial usage

Blog: Large Enough

Weights: mistralai/Mistral-Large-Instruct-2407 · Hugging Face

1/N pic.twitter.com/k2o7FbmYiE

— Devendra Chaplot (@dchaplot) July 24, 2024

While having a lower number of parameters — or internal model settings that guide its performance — than Llama 3.1’s 405 billion, it still nears the former’s performance.


Available on the company’s main platform and via cloud partners, Mistral Large 2 builds on the original Large model and brings advanced multilingual capabilities with improved performance across reasoning, code generation and mathematics.

It is being hailed as a GPT-4 class model with performance closely matching GPT-4o, Llama 3.1-405 and Anthropic’s Claude 3.5 Sonnet across several benchmarks.

Mistral notes the offering continues to “push the boundaries of cost efficiency, speed and performance” while giving users new features, including advanced function calling and retrieval, to build high-performing AI applications.

However, it’s important to note that this isn’t a one-off move designed to cut off the AI hype stirred by Meta or OpenAI. Mistral has been moving aggressively in the domain, raising large rounds, launching new task-specific models (including those for coding and mathematics) and partnering with industry giants to expand its reach.

Mistral Large 2: What to expect?​

Back in February, when Mistral launched the original Large model with a context window of 32,000 tokens, it claimed that the offering had “a nuanced understanding of grammar and cultural context” and could reason with and generate text with native fluency across different languages, including English, French, Spanish, German and Italian.

The new version of the model builds on this with a larger 128,000 context window — matching OpenAI’s GPT-4o and GPT-4o mini and Meta’s Llama 3.1.

It further boasts support for dozens of new languages, including the original ones as well as Portuguese, Arabic, Hindi, Russian, Chinese, Japanese and Korean.

Mistral says the model has large reasoning capabilities and can easily handle complex tasks like synthetic text generation, code generation and RAG.

High performance on third-party benchmarks and improved coding capability​

On the Multilingual MMLU benchmark covering different languages, Mistral Large 2 performed on par with Meta’s all-new Llama 3.1-405B while delivering more significant cost benefits due to its smaller size.

“Mistral Large 2 is designed for single-node inference with long-context applications in mind – its size of 123 billion parameters allows it to run at large throughput on a single node,” the company noted in a blog post.

Mistral Large 2 on Multilingual MMLU

But, that’s not the only benefit.

The original Large model did not do well on coding tasks, which Mistral seems to have remediated after training the latest version on large chunks of code.

The new model can generate code in 80+ programming languages, including Python, Java, C, C++, JavaScript and Bash, with a very high level of accuracy (according to the average from MultiPL-E benchmark).

On HumanEval and HumanEval Plus benchmarks for code generation, it outperformed Claude 3.5 Sonnet and Claude 3 Opus, while sitting just behind GPT-4o. Similarly, across Mathematics-focused benchmarks – GSM8K and Math Instruct – it grabbed the second spot.

Focus on instruction-following with minimized hallucinations​

Given the rise of AI adoption by enterprises, Mistral has also focused on minimizing the hallucination of Mistral Large by fine-tuning the model to be more cautious and selective when responding. If it doesn’t have sufficient information to back an answer, it will simply tell that to the user, ensuring full transparency.

Further, the company has improved the model’s instruction-following capabilities, making it better at following user guidelines and handling long multi-turn conversations. It has even been tuned to provide succinct and to-the-point answers wherever possible — which can come in handy in enterprise settings.

Currently, the company is providing access to Mistral Large 2 through its API endpoint platform as well as via cloud platforms such as Google Vertex AI, Amazon Bedrock, Azure AI Studio and IBM WatsonX. Users can even test it via the company’s chatbot to see how it works in the world.

 

[bnew]

Microsoft unveils serverless fine-tuning for its Phi-3 small language model

While it's a clear win for developers looking to stay in the Microsoft ecosystem, it's also a notable competitor to Microsoft's own ally.

venturebeat.com

Microsoft unveils serverless fine-tuning for its Phi-3 small language model​

Carl Franzen@carlfranzen

July 25, 2024 3:31 PM

Credit: VentureBeat made with Midjourney V6

---

Microsoft is a major backer and partner of OpenAI, but that doesn’t mean it wants to let the latter company run away with the generative AI ballgame.

As proof of that, today Microsoft announced a new way to fine-tune its Phi-3 small language model without developers having to manage their own servers, and for free (initially).

Fine-tuning refers to the process of adapting an AI model through system prompts or adjusting its underlying weights (parameters) to make it behave in different and more optimal ways for specific use cases and end users, even adding new capabilities.

What is Phi-3?​

The company unveiled Phi-3, a 3 billion parameter model, back in April as a low-cost, enterprise grade option for third-party developers to build new applications and software atop of.

While significantly smaller than most other leading language models (Meta’s Llama 3.1 for instance, comes in a 405 billion parameter flavor — parameters being the “settings” that guide the neural network’s processing and responses), Phi-3 performed on the level of OpenAI’s GPT-3.5 model, according to comments provided at that time to VentureBeat by Sébastien Bubeck, Vice President of Microsoft generative AI.

Specifically, Phi-3 was designed to offer affordable performance on coding, common sense reasoning, and general knowledge.

It’s now a whole family consisting of 6 separate models with different numbers of parameters and context lengths (the amount of tokens, or numerical representations of data) the user can provide in a single input, the latter ranging from 4,000 to 128,000 — with costs ranging from $0.0003 USD per 1,000 input tokens to $0.0005 USD/1K input tokens.

However, put into the more typical “per million” token pricing, it comes out to $0.3/$0.9 per 1 million tokens to start, exactly double OpenAI’s new GPT-4o mini pricing for input and about 1.5 times as expensive for output tokens.

Phi-3 was designed to be safe for enterprises to use with guardrails to reduce bias and toxicity. Even back when it was first announced, Microsoft’s Bubeck promoted its capability to be fine-tuned for specific enterprise use cases.

“You can bring in your data and fine-tune this general model, and get amazing performance on narrow verticals,” he told us.

But at that point, there was no serverless option to fine-tune it: if you wanted to do it, you had to set up your own Microsoft Azure server or download the model and run it on your own local machine, which may not have enough space.

Serverless fine-tuning unlocks new options​

Today, however, Microsoft announced the general public availability of its “Models-as-a-Service (serverless endpoint)” in its Azure AI development platform.

It also announced that “Phi-3-small is now available via a serverless endpoint so developers can quickly and easily get started with AI development without having to manage underlying infrastructure.”

Phi-3-vision, which can handle imagery inputs “will soon be available via a serverless endpoint” as well, according to Microsoft’s blog post.

But those models are simply available “as is” through Microsoft’s Azure AI development platform. Developers can build apps atop them, but they can’t create their own versions of the models tuned to their own use cases.

For developers looking to do that, Microsoft says they should turn to the Phi-3-mini and Phi-3-medium, which can be fine-tuned with third-party “data to build AI experiences that are more relevant to their users, safely, and economically.”

“Given their small compute footprint, cloud and edge compatibility, Phi-3 models are well suited for fine-tuning to improve base model performance across a variety of scenarios including learning a new skill or a task (e.g. tutoring) or enhancing consistency and quality of the response (e.g. tone or style of responses in chat/Q&A),” the company writes.

Specifically, Microsoft states that the educational software company Khan Academy is already using a fine-tuned Phi-3 to benchmark the performance of its Khanmigo for Teachers powered by Microsoft’s Azure OpenAI Service.

A new price and capability war for enterprise AI developers​

The pricing for serverless fine-tuning of Phi-3-mini-4k-instruct starts at $0.004 per 1,000 tokens ($4 per 1 million tokens), while no pricing has been listed yet for the medium model.

While it’s a clear win for developers looking to stay in the Microsoft ecosystem, it’s also a notable competitor to Microsoft’s own ally OpenAI’s efforts to capture enterprise AI developers.

And OpenAI just days ago announced free fine-tuning of GPT-4o mini up to 2 million tokens per day through September 23rd, for so-called “Tier 4 and 5” users of its application programming interface (API), or those who spend at least $250 or $1000 on API credits.

Coming also on the heels of Meta’s release of the open source Llama 3.1 family and Mistral’s new Mistral Large 2 model, both of which can also be fine tuned for different uses, it’s clear the race to offer compelling AI options for enterprise development is in full swing — and AI providers are courting developers with both small and big models.

 

[bnew]

AI models rank their own safety in OpenAI’s new alignment research

Rules-based Rewards, a method from OpenAI that automates safety scoring, lets developers create clear-cut safety instructions for AI model fine-tuning.

venturebeat.com

AI models rank their own safety in OpenAI’s new alignment research​

Emilia David@miyadavid

July 24, 2024 9:00 AM

Image Credit: VentureBeat via DALL-E, OpenAI

Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More

---

OpenAI announced a new way to teach AI models to align with safety policies called Rules Based Rewards.

According to Lilian Weng, head of safety systems at OpenAI, Rules-Based Rewards (RBR) automate some model fine-tuning and cut down the time required to ensure a model does not give unintended results.

“Traditionally, we rely on reinforcement learning from human feedback as the default alignment training to train models, and it works,” Weng said in an interview. “But in practice, the challenge we’re facing is that we spend a lot of time discussing the nuances of the policy, and by the end, the policy may have already evolved.”

Weng referred to reinforcement learning from human feedback, which asks humans to prompt a model and rate its answers based on accuracy or which version they prefer. If a model is not meant to respond a certain way—for example, sound friendly or refuse to answer “unsafe” requests like asking for something dangerous—human evaluators can also score its response to see if it follows policies.

With RBR, OpenAI said safety and policy teams use an AI model that scores responses based on how closely they adhere to a set of rules created by the teams.

For example, the model development team of a mental health app wants the AI model to refuse unsafe prompts but in a non-judgemental manner, along with reminders to seek help if needed. They would have to create three rules for the model to follow: first, it needs to reject the request; second, sound non-judgemental; and third, use encouraging words for users to seek help.

The RBR model looks at responses from the mental health model, maps it to the three basic rules, and determines if these check the boxes of the rules. Weng said the results from testing models using RBR are comparable to human-led reinforcement learning.

Of course, ensuring AI models respond within specific parameters is difficult, and when the models fail, it creates controversy. In February, Google said it overcorrected Gemini’s image generation restriction after the model continually refused to generate photos of white people and created ahistorical images instead.

Reducing human subjectivity​

For many, myself included, the idea of models being in charge of another model’s safety raises concerns. But Weng said RBR actually cuts down on subjectivity, an issue that human evaluators often face.

“My counterpoint would be even when you’re working with human trainers, the more ambiguous or murky your instruction is, the lower quality data you’ll get,” she said. “If you say pick which one is safer, then that’s not really an instruction people can follow because safe is subjective, so you narrow down your instructions, and in the end, you’re left with the same rules we give to a model.”

OpenAI understands that RBR could reduce human oversight and presents ethical considerations that include potentially increasing bias in the model. In a blog post, the company said researchers “should carefully design RBRs to ensure fairness and accuracy and consider using a combination of RBRs and human feedback.”

RBR may have difficulty with tasks designed to be subjective, like writing or anything creative.

OpenAI began exploring RBR methods while developing GPT-4, though Weng said RBR has greatly evolved since then.

OpenAI has been questioned about its commitment to safety. In March, Jan Leike, a former researcher and leader of the company’s Superalignment team, blasted it by posting that “safety culture and processes have taken a backseat to shiny products.” Co-founder and chief scientist Ilya Sutskever, who co-led the Superalignment team with Leike, also resigned from OpenAI. Sutskever has since started a new company focused on safe AI systems.

 

[bnew]

Yann LeCun says Meta AI ‘quickly becoming most used’ assistant, challenging OpenAI’s dominance

Meta challenges OpenAI with Llama 3.1, an open-source AI model rivaling GPT-4o, as Yann LeCun claims Meta AI is becoming the most widely used AI assistant.

venturebeat.com

Yann LeCun says Meta AI ‘quickly becoming most used’ assistant, challenging OpenAI’s dominance​

Michael Nuñez@MichaelFNunez

July 23, 2024 2:26 PM

Credit: VentureBeat made with Midjourney

Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More

---

Meta Platforms has thrown down the gauntlet in the AI race today with the release of Llama 3.1, its most sophisticated artificial intelligence model to date.

This advanced model now powers Meta AI, the company’s AI assistant, which has been strategically deployed across its suite of platforms including WhatsApp, Messenger, Instagram, Facebook, and Ray-Ban Meta, with plans to extend to Meta Quest next month. The widespread implementation of Llama 3.1 potentially places advanced AI capabilities at the fingertips of billions of users globally.

The move represents a direct challenge to industry leaders OpenAI and Anthropic, particularly targeting OpenAI’s market-leading position. It also underscores Meta’s commitment to open-source development, marking a major escalation in the AI competition.

Llama 3.1 now powers Meta AI, which is quickly becoming the most widely used AI assistant.

Meta AI can be accessed through WhatsApp, Messenger, Instagram, Facebook, Ray-Ban Meta, and next month in Meta Quest.

It answers questions, summarizes long documents, helps you code or do…

— Yann LeCun (@ylecun) July 23, 2024

Yann LeCun, Meta’s chief AI scientist, made a bold proclamation on X.com following the release this morning that caught many in the AI community off guard. “Llama 3.1 now powers Meta AI, which is quickly becoming the most widely used AI assistant,” LeCun said, directly challenging the supremacy of OpenAI’s ChatGPT, which has thus far dominated the AI assistant market.

If substantiated, LeCun’s assertion could herald a major shift in the AI landscape, potentially reshaping the future of AI accessibility and development.

Open-source vs. Closed-source: Meta’s disruptive strategy in the AI market​

The centerpiece of Meta’s release is the Llama 3.1 405B model, featuring 405 billion parameters. The company boldly contends that this model’s performance rivals that of leading closed-source models, including OpenAI’s GPT-4o, across various tasks. Meta’s decision to make such a powerful model openly available stands in stark contrast to the proprietary approaches of its competitors, particularly OpenAI.

This release comes at a critical juncture for Meta, following a $200 billion market value loss earlier this year. CEO Mark Zuckerberg has pivoted the company’s focus towards AI, moving away from its previous emphasis on the metaverse. “Open source will ensure that more people around the world have access to the benefits and opportunities of AI,” Zuckerberg said, in what appears to be a direct challenge to OpenAI’s business model.

Wall Street analysts have expressed skepticism about Meta’s open-source strategy, questioning its potential for monetization, especially when compared to OpenAI’s reported $3.4 billion annualized revenue. However, the tech community has largely welcomed the move, seeing it as a catalyst for innovation and wider AI access.

Our Llama 3.1 405B is now openly available! After a year of dedicated effort, from project planning to launch reviews, we are thrilled to open-source the Llama 3 herd of models and share our findings through the paper:

?Llama 3.1 405B, continuously trained with a 128K context… pic.twitter.com/RwhedAluSM

— Aston Zhang (@astonzhangAZ) July 23, 2024

AI arms race heats up: Implications for innovation, safety, and market leadership​

The new model boasts improvements including an extended context length of 128,000 tokens, enhanced multilingual capabilities, and improved reasoning. Meta has also introduced the “Llama Stack,” a set of standardized interfaces aimed at simplifying development with Llama models, potentially making it easier for developers to switch from OpenAI’s tools.

While the release has generated excitement in the AI community, it also raises concerns about potential misuse. Meta claims to have implemented robust safety measures, but the long-term implications of widely available advanced AI remain a topic of debate among experts.

Why are FTC & DOJ issuing statements w/ EU competition authorities discussing "risks" in the blazingly competitive, U.S.-built AI ecosystem? And on the same day that Meta turbocharges disruptive innovation with the first-ever frontier-level open source AI model? A ? pic.twitter.com/vrItR28YIo

— Neil Chilson ?? ? (@neil_chilson) July 23, 2024

As the AI race intensifies, Meta’s latest move positions the company as a formidable competitor in a field previously dominated by OpenAI and Anthropic. The success of Llama 3.1 could potentially reshape the AI industry, influencing everything from market dynamics to development methodologies.

The tech industry is closely watching this development, with many speculating on how OpenAI and other AI leaders will respond to Meta’s direct challenge. As the competition heats up, the implications for AI accessibility, innovation, and market leadership remain to be seen, with OpenAI’s dominant position now under serious threat.

 

[bnew]

Nvidia’s latest AI offering could spark a custom model gold rush

Nvidia launches AI Foundry service, enabling businesses to create custom AI models with increased accuracy and control, potentially revolutionizing enterprise AI adoption.

venturebeat.com

Nvidia’s latest AI offering could spark a custom model gold rush​

Michael Nuñez@MichaelFNunez

July 24, 2024 3:27 PM

Credit: VentureBeat made with Midjourney

---

Nvidia quietly unveiled its new AI Foundry service on Tuesday, aiming to help businesses create and deploy custom large language models tailored to their specific needs. The move signals Nvidia’s push to capture a larger share of the booming enterprise AI market.

The AI Foundry combines Nvidia’s hardware, software tools, and expertise to enable companies to develop customized versions of popular open-source models like Meta’s recently released Llama 3.1. This service arrives as businesses increasingly seek to harness the power of generative AI while maintaining control over their data and applications.

“This is really the moment we’ve been waiting for,” said Kari Briski, Nvidia’s VP of AI Software, in a call with VentureBeat. “Enterprises scrambled to learn about generative AI. But something else happened that was probably equally important: the availability of open models.”

Customization drives accuracy: How Nvidia’s AI Foundry boosts model performance​

Nvidia’s new offering aims to simplify the complex process of adapting these open models for specific business use cases. The company claims significant improvements in model performance through customization. “We’ve seen almost a ten point increase in accuracy by simply customizing models,” Briski explained.

The AI Foundry service provides access to a vast array of pre-trained models, high-performance computing resources through Nvidia’s DGX Cloud, and NeMo toolkit for model customization and evaluation. Expert guidance from Nvidia’s AI specialists is also part of the package.

“We provide the infrastructure and the tools for other companies to develop and customize AI models,” Briski said. “Enterprises bring their data, we have DGX cloud that has capacity across many of our cloud partners.”

NIM: Nvidia’s unique approach to AI model deployment​

Alongside the AI Foundry, Nvidia introduced NIM (Nvidia Inference Microservices), which packages customized models into containerized, API-accessible formats for easy deployment. This development represents a significant milestone for the company. “NIM is a model, a customized model and a container accessed by standard API,” Briski said. “This is the culmination of years of work and research that we’ve done.”

Industry analysts view this move as a strategic expansion of Nvidia’s AI offerings, potentially opening up new revenue streams beyond its core GPU business. The company is positioning itself as a full-stack AI solutions provider, not just a hardware manufacturer.

Enterprise AI adoption: Nvidia’s strategic bet on custom models​

The timing of Nvidia’s announcement is particularly significant, happening the same day as Meta’s Llama 3.1 release and amid growing concerns about AI safety and governance. By offering a service that allows companies to create and control their own AI models, Nvidia may be tapping into a market of enterprises that want the benefits of advanced AI without the risks associated with using public, general-purpose models.

However, the long-term implications of widespread custom AI model deployment remain unclear. Potential challenges include fragmentation of AI capabilities across industries and the difficulty of maintaining consistent standards for AI safety and ethics.

As competition in the AI sector intensifies, Nvidia’s AI Foundry represents a significant bet on the future of enterprise AI adoption. The success of this gamble will largely depend on how effectively businesses can leverage these custom models to drive real-world value and innovation in their respective industries.

 

[bnew]

Free Gemini users can finally chat in a flash

Google is making Gemini 1.5 Flash available for free users of the Gemini chatbot. The company is also adding related links to responses.

venturebeat.com

Free Gemini users can finally chat in a flash​

Emilia David@miyadavid

July 25, 2024 9:00 AM

Sir Demis Hassabis introduces Gemini 1.5 Flash. Image credit: Screenshot

---

Google made several updates to the free version of its Gemini chatbot, including making its low-latency multimodal model Gemini 1.5 Flash available and adding more source links to reduce hallucinations.

Gemini 1.5 Flash, previously only available to developers, is best suited for tasks requiring quick responses, such as answering customer queries. Google announced the model during its annual developer conference, Google I/O, in May but has since opened it up to the public.

The model has a large context window, referring to how much information or words it processes at a time, of around 1 million tokens. Google said Gemini 1.5 Flash on the Gemini chatbot will have a context window of 32K tokens. A large context window allows for more complex questions and longer back-and-forth conversations.

To take advantage of this, Google is updating the free version of Gemini to handle file uploads from Google Drive or devices. This has been a feature in Gemini Advanced, the paid version of the chatbot.

When it first launched, Google claimed Gemini 1.5 Flash was 40% faster than OpenAI’s fast model GPT-3.5 Turbo. Gemini 1.5 Flash is not a small model like the Gemma family of Google models; instead, it is trained with the same data as Gemini 1.5 Pro.

Gemini 1.5 Flash will be available on both mobile and desktop versions of Gemini. It can be accessed in more than 230 countries and territories and in 40 languages.

Reducing hallucinations with links​

Hallucinations continue to be a problem for AI models. Google is following the lead of other model providers and chatbots by adding related links to prompts asking for information. The idea is to show the AI models did not create the information without reference.

“Starting today for English language prompts in certain countries, you can access this additional information on topics directly within Gemini’s responses. Just click on the chip at the end of a paragraph to see websites where you can dive deeper on a certain topic,” Google said in a blog post.

The company said Gemini will add links to the relevant email if the information is in an email.

Google will also add a double-check feature that “verifies responses by using Google Search to highlight which statements are corroborated or contradicted on the web.”

Google is not the only company that adds links for attribution in line with the responses on a chatbot. ChatGPT and Perplexity regularly add citations and links to websites where they find information.

However, a report from Nieman Labs found that the chatbots hallucinated some links, in some cases attaching links to news stories that do not exist or are completely unrelated.

 

[bnew]

Forget coding bootcamps: Airtable’s AI can build your app in seconds

Airtable's Cobuilder transform app development with AI, allowing non-coders to create custom applications in seconds using natural language prompts.

venturebeat.com

Forget coding bootcamps: Airtable’s AI can build your app in seconds​

Michael Nuñez@MichaelFNunez

July 25, 2024 11:00 AM

Credit: VentureBeat made with Midjourney

---

Airtable, the $11 billion no-code platform unicorn, has unveiled Cobuilder, an AI-powered tool that generates customizable applications in seconds using natural language prompts. This launch could reshape the landscape of enterprise software development by allowing non-technical employees to build complex applications without coding knowledge.

Kelly O’Shaughnessy, head of core product and product lead for Airtable Cobuilder, explained the tool’s significance in an interview with VentureBeat. “Cobuilder is the fastest way to build no-code applications, making it possible to create customizable apps with natural language in just seconds,” she said. “Paired with the vast amounts of knowledge across industries, use cases, and business concepts that today’s LLMs have, Cobuilder helps anyone take an idea from concept to reality in seconds and transform their workflows.”

A screenshot of Airtable’s new Cobuilder tool shows how it generates a customized app for managing the release of a women’s skateboarding shoe. The interface displays a kanban board with various stages of the product launch, demonstrating the AI’s ability to create complex project management solutions from simple natural language inputs. (Image Credit: Airtable)

AI-powered app generation: How Cobuilder transforms ideas into software​

The technology behind Cobuilder uses large language models (LLMs) to interpret user prompts and generate appropriate application structures. O’Shaughnessy elaborated on the process, telling VentureBeat, “Cobuilder generates an application by analyzing a user’s natural language prompt and matching the request with relevant publicly available data the LLM provider has access to.”

This approach could significantly reduce the time and resources required for application development, a process that traditionally involves multiple stakeholders and can take months or even years. O’Shaughnessy highlighted this advantage, citing Airtable CEO Howie Liu’s LinkedIn post on the new product launch, where he said, “Traditional software development is expensive and slow, but the bigger problem is the disconnect that happens when there are layers of separation between the technical software developers building an app and the business stakeholders who understand the actual requirements for the app.”

The adoption of AI-generated applications in enterprise environments raises questions about data privacy and security. Addressing these concerns, O’Shaughnessy told VentureBeat, “Airtable protects the privacy and security of customers’ data. No customer data is used to train current or future LLMs.”

Balancing innovation and limitations: The current state of AI-generated apps​

While Cobuilder represents a leap forward in no-code development, it currently relies on publicly available data and user-provided information to create applications. Airtable plans to enhance Cobuilder’s capabilities, including the ability to incorporate existing company data from Airtable and embed AI automations within generated apps.

The launch of Cobuilder is part of Airtable’s broader strategy to integrate AI across its platform. Earlier this year, the company introduced Airtable AI, which has already seen adoption by major clients like AWS. Future plans include expanding document extraction capabilities and enabling AI-powered internet search integration.

For Airtable, this move represents a significant bet on the future of enterprise software development. As businesses increasingly seek ways to empower non-technical staff and reduce reliance on traditional development processes, tools like Cobuilder could become increasingly attractive.

The future of enterprise software: Airtable’s vision for AI-driven development​

O’Shaughnessy envisions a transformative impact. “This combo of no-code and AI unlocks the ability for non-experts and non-developers to describe the workflow they need in plain language—as though having a conversation with a developer—and then Cobuilder helps create an app with the best design and operational structures in seconds.”

As Airtable continues to push the boundaries of no-code development with AI integration, it positions itself at the forefront of a potential new era in enterprise software creation. The success of Cobuilder could not only solidify Airtable’s position in the market but also potentially reshape how businesses approach software development in the coming years.

 

[bnew]

Runway faces backlash after report of copying AI video training data from YouTube

Runway revealed Gen-3 Alpha, an early version of the software, to acclaim for its realism, last month, and began allowing the public...

venturebeat.com

Runway faces backlash after report of copying AI video training data from YouTube​

Carl Franzen@carlfranzen

July 25, 2024 11:49 AM

Credit: VentureBeat made with Midjourney V6

---

Runway, a multi-hundred million dollar funded startup focused on AI video software and models that is backed by Google, among others, is in hot water from creators following a report today by 404 Media on a spreadsheet allegedly showing it undertook an effort to copy data from thousands of YouTube videos.

404 Media reports that a former employee of Runway leaked it a company spreadsheet allegedly showing its plans to categorize, tag, and train on “YouTube channels of thousands of media and entertainment companies, including The New Yorker, VICE News, Pixar, Disney, Netflix, Sony, and many other,” and that this data informed a product called “Jupiter,” which 404 says is Runway’s Gen-3 AI video creation model.

Individual YouTubers with large followings such as “Casey Neistat, Sam Kolder, Benjamin Hardman, Marques Brownlee” also were included in the spreadsheet.

We’ve reached out to Runway to verify the authenticity of the spreadsheet and will update when we hear back.

​

Fruit from the poisonous tree behind Gen-3 Alpha?​

Runway revealed Gen-3 Alpha, an early version of the software, to acclaim for its realism, last month, and began allowing the public to use it a few weeks ago.

404 Media published a redacted Google Sheets copy of the alleged Runway spreadsheet online as a link within its article, showing more than 3,900 individual YouTube channels and a column with hashtags of different content contained therein.

Another tab of the spreadsheet labeled “high_camera_movement” includes more than 177 distinct YouTube accounts.

​

Rubbing creators and critics the wrong way​

404 Media notes in its report that it “couldn’t confirm that every single video included in the spreadsheet was used to train Gen-3—it’s possible that some content was filtered out later or that not every single link on the spreadsheet was scraped,” but the existence of the spreadsheet itself and the implication that all or any of these YouTube videos may have been copied, downloaded, or otherwise analyzed by Runway engineers and/or machine learning algorithms to inform its Gen-3 Alpha model (or any other product for that matter) has rubbed many creators and critics of generative AI the wrong way.

Influential tech reviewer YouTuber Marques Brownlee a.k.a. MKBHD posted on X “well well well” and included a melting smiley face emoji. Brownlee has been critical in the past of others training AI on his videos.

Well well well. Runway AI video generator was trained on YouTube videos without permission, including 1600+ MKBHD videos ?AI Video Generator Runway Trained on Thousands of YouTube Videos Without Permission

— Marques Brownlee (@MKBHD) July 25, 2024

Yet he’s also expressed excitement and enthusiasm for AI video technology such as OpenAI’s Sora in a prior video.

Ed Newton-Rex, founder and CEO of the ethical AI certification startup Fairly Trained, has posted several times on X highlighting the various notable names included in the alleged Runway spreadsheet, among them YouTube channels for musician Taylor Swift and filmmaker Wes Anderson.

Here are some of the entries in Runway's spreadsheet entitled "Video sourcing", unearthed by @404mediaco … ?

1. A playlist of all Taylor Swift's music videos x.com pic.twitter.com/7EG75eHaaP

— Ed Newton-Rex (@ednewtonrex) July 25, 2024

YouTuber Omni or “Lay It Omni” called the spreadsheet “INSANE” in an X post and accused Runway of theft.

guys this is actually INSANE. a former employee of a multi-billion dollar company, Runway, confirmed that they mass downloaded YouTube videos in order to feed their AI. there's a spreadsheet with NOTES showing HOW they swiped videos. Nintendo was on the list. x.com

— Omni ️ (@InfernoOmni) July 25, 2024

THEY STOLE FROM MIYAZAKI?? AND USED KISSANIME TO GET ANIME VIDEOS OH MY GOD pic.twitter.com/042UNhzJcN

— Omni ️ (@InfernoOmni) July 25, 2024

Even AI filmmakers who have created with Runway’s tools in the past including Dustin Hollywoodhave expressed criticism towards the company for what they view as theft.

I feel a shyt storm coming about GEN3.. ??

When are companies gonna learn, purchase your data, create paid artist programs to create and feed you data. DONT fukkING STEAL DATA. Damn.

No one one learns because of greed. If you think people are not working on ways/institutions…

— Dustin Hollywood (@dustinhollywood) July 25, 2024

Yet as I pointed out in a reply on X to Hollywood, multiple companies have already been accused or found to have used copyrighted videos without express permission or authorization or payment in training their models.

Indeed, just recently, Wired magazine (where my wife works as Editor-in-Chief)published a piece in conjunction with Proof News that found such big names as Apple, Nvidia, and the AI startup Anthropic (maker of Claude 3 Sonnet and Claude family of models) also trained AI models on YouTube Video transcripts without authorization.

My take is that scraping and training, while controversial, is legal and supported by the precedent set by Google in scraping the web and indexing it for search. But we’ll see if this holds up in court, asRunway is already among one of many AI companies being sued by creators for training on their data without permission or compensation. And in the court of public opinion, Runway appears to have taken a big hit today.