Large Language Models News & Discussions

[bnew]

Suno

Suno is building a future where anyone can make great music.

www.suno.ai

suno/bark · Hugging Face

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

huggingface.co

GitHub - suno-ai/bark: 🔊 Text-Prompted Generative Audio Model

🔊 Text-Prompted Generative Audio Model. Contribute to suno-ai/bark development by creating an account on GitHub.

github.com

Notice: Bark is Suno's open-source text-to-speech+ model. If you are looking for our new text-to-music model, Chirp, have a look at our Chirp Examples Page and join us on Discord.

Bark​

Examples • Suno Studio Waitlist • Updates • How to Use • Installation • FAQ

​

Bark is a transformer-based text-to-audio model created by Suno. Bark can generate highly realistic, multilingual speech as well as other audio - including music, background noise and simple sound effects. The model can also produce nonverbal communications like laughing, sighing and crying. To support the research community, we are providing access to pretrained model checkpoints, which are ready for inference and available for commercial use.

DEMO:

Bark - a Hugging Face Space by suno

Discover amazing ML apps made by the community

huggingface.co

 

[bnew]

Apple’s latest AI research could completely transform your iPhone

Apple researchers have introduced new techniques to create photorealistic 3D avatars from video and enable advanced AI systems to run efficiently on devices with limited memory, such as an iPhone or iPad.

venturebeat.com

Apple’s latest AI research could completely transform your iPhone​

Michael Nuñez
@MichaelFNunez

December 20, 2023 1:01 PM

Credit: VentureBeat made with Midjourney

Are you ready to bring more awareness to your brand? Consider becoming a sponsor for The AI Impact Tour. Learn more about the opportunities here.

---

Apple, a company practically synonymous with technological innovation, has once again positioned itself at the forefront of the AI revolution.

The Cupertino, Calif.-based company recently announced significant strides in artificial intelligence research through two new papers introducing new techniques for 3D avatars and efficient language model inference. The advancements could enable more immersive visual experiences and allow complex AI systems to run on consumer devices such as the iPhone and iPad.

In the first research paper, Apple scientists propose HUGS (Human Gaussian Splats) to generate animated 3D avatars from short monocular videos (i.e. videos taken from a single camera). “Our method takes only a monocular video with a small number of (50-100) frames, and it automatically learns to disentangle the static scene and a fully animatable human avatar within 30 minutes,” said lead author Muhammed Kocabas.

The training video (left upper), the reconstructed canonical human avatar (right upper), the reconstructed scene model (left bottom), and the animated reposed human together with the scene (right bottom). (Credit: Apple)

HUGS represents both the human and background scene using 3D Gaussian splatting, an efficient rendering technique. The human model is initialized from a statistical body shape model called SMPL. But HUGS allows the Gaussians to deviate, enabling capture of details like clothing and hair.

A novel neural deformation module animates the Gaussians in a realistic fashion using linear blend skinning. This coordinated movement avoids artifacts while reposing the avatar. According to Kocabas, HUGS “enables novel-pose synthesis of human and novel view synthesis of both the human and the scene.”

Compared to previous avatar generation methods, HUGS is up to 100 times faster in training and rendering. The researchers demonstrate photorealistic results after optimizing the system for just 30 minutes on a typical gaming GPU. HUGS also outperforms state-of-the-art techniques like Vid2Avatar and NeuMan on 3D reconstruction quality.

The new technology lets people put different digital characters, or “avatars,” into a new scene using just one video of the person and the place. This can be done quickly, with the image updating 60 times every second to make it look smooth and realistic. (Credit: Apple)

The new 3D modeling capabilitiy is a really impressive achievement from Apple researchers. The real-time performance and ability to create avatars from in-the-wild videos could unlock new possibilities for virtual try-on, telepresence, and synthetic media in the relatively near future. Imagine the possibilities if you could create novel 3D scenes like this right on your iPhone camera!

Bridging the memory gap in AI inference​

In the second paper, Apple researchers tackled a key challenge in deploying large language models (LLMs) on devices with limited memory. Modern natural language models like GPT-4 contain hundreds of billions of parameters, making inference expensive on consumer hardware.

The proposed system minimizes data transfer from flash storage into scarce DRAM during inference. “Our method involves constructing an inference cost model that harmonizes with the flash memory behavior, guiding us to optimize in two critical areas: reducing the volume of data transferred from flash and reading data in larger, more contiguous chunks,” explained lead author Keivan Alizadeh.

Two main techniques are introduced. “Windowing” reuses activations from recent inferences, while “row-column bundling” reads larger blocks of data by storing rows and columns together. On an Apple M1 Max CPU, these methods improve inference latency by 4-5x compared to naive loading. On a GPU, the speedup reaches 20-25x.

“This breakthrough is particularly crucial for deploying advanced LLMs in resource-limited environments, thereby expanding their applicability and accessibility,” said co-author Mehrdad Farajtabar. The optimizations could soon allow complex AI assistants and chatbots to run smoothly on iPhone, iPads, and other mobile devices.

Apple’s strategic vision​

Both papers demonstrate Apple’s growing leadership in AI research and applications. While promising, experts caution that Apple will need to exercise great care and responsibility when incorporating these technologies into consumer products. From privacy protection to mitigating misuse, the societal impact must be considered.

As Apple potentially integrates these innovations into its product lineup, it’s clear that the company is not just enhancing its devices but also anticipating the future needs of AI-infused services. By allowing more complex AI models to run on devices with limited memory, Apple is potentially setting the stage for a new class of applications and services that leverage the power of LLMs in a way that was previously unfeasible.

Furthermore, by publishing this research, Apple is contributing to the broader AI community, which could stimulate further advancements in the field. It’s a move that reflects Apple’s confidence in its position as a tech leader and its commitment to pushing the boundaries of what’s possible.

If applied judiciously, Apple’s latest innovations could take artificial intelligence to the next level. Photorealistic digital avatars and powerful AI assistants on portable devices once seemed far off — but thanks to Apple’s scientists, the future is rapidly becoming reality.

---

Apple AI researchers publish papers on using human gaussian splats to generate animated 3D avatars from videos and deploying LLMs on devices with limited memory

By Michael Nuñez / VentureBeat. View the full context on Techmeme.

www.techmeme.com

 

[bnew]

Apple Develops Breakthrough Method for Running LLMs on iPhones

Apple GPT in your pocket? It could be a reality sooner than you think. Apple AI researchers say they have made a key breakthrough in deploying large...

www.macrumors.com

Apple Develops Breakthrough Method for Running LLMs on iPhones​

Thursday December 21, 2023 2:26 am PST by Tim Hardwick

Apple GPT in your pocket? It could be a reality sooner than you think. Apple AI researchers say they have made a key breakthrough in deploying large language models (LLMs) on iPhones and other Apple devices with limited memory by inventing an innovative flash memory utilization technique.

LLMs and Memory Constraints​

LLM-based chatbots like ChatGPT and Claude are incredibly data and memory-intensive, typically requiring vast amounts of memory to function, which is a challenge for devices like iPhones that have limited memory capacity. To tackle this issue, Apple researchers have developed a novel technique that uses flash memory – the same memory where your apps and photos live – to store the AI model's data.

Storing AI on Flash Memory​

In a new research paper titled "LLM in a flash: Efficient Large Language Model Inference with Limited Memory," the authors note that flash storage is more abundant in mobile devices than the RAM traditionally used for running LLMs. Their method cleverly bypasses the limitation using two key techniques that minimize data transfer and maximize flash memory throughput:

1. Windowing: Think of this as a recycling method. Instead of loading new data every time, the AI model reuses some of the data it already processed. This reduces the need for constant memory fetching, making the process faster and smoother.
2. Row-Column Bundling: This technique is like reading a book in larger chunks instead of one word at a time. By grouping data more efficiently, it can be read faster from the flash memory, speeding up the AI's ability to understand and generate language.
   

The combination of these methods allows AI models to run up to twice the size of the iPhone's available memory, according to the paper. This translates to a 4-5 times increase in speed on standard processors (CPUs) and an impressive 20-25 times faster on graphics processors (GPUs). "This breakthrough is particularly crucial for deploying advanced LLMs in resource-limited environments, thereby expanding their applicability and accessibility," write the authors.

Faster AI on iPhone​

The breakthrough in AI efficiency opens new possibilities for future iPhones, such as more advanced Siri capabilities, real-time language translation, and sophisticated AI-driven features in photography and augmented reality. The technology also sets the stage for iPhones to run complex AI assistants and chatbots on-device, something Apple is already said to be working on.

Apple's work on generative AI could eventually be incorporated into its ‌Siri‌ voice assistant. Apple in February 2023 held an AI summit and briefed employees on its large language model work. According to Bloomberg, Apple is aiming for a smarter version of Siri that's deeply integrated with AI. Apple is planning to update the way that ‌Siri‌ interacts with the Messages app, allowing users to field complex questions and auto-complete sentences more effectively. Beyond that, Apple is rumored to be planning to add AI to as many Apple apps as possible.

Apple GPT​

Apple is reportedly developing its own generative AI model called "Ajax". Designed to rival the likes of OpenAI's GPT-3 and GPT-4, Ajax operates on 200 billion parameters, suggesting a high level of complexity and capability in language understanding and generation. Internally known as "Apple GPT," Ajax aims to unify machine learning development across Apple, suggesting a broader strategy to integrate AI more deeply into Apple's ecosystem.

As of the latest reports, Ajax is considered more capable than the earlier generation ChatGPT 3.5. However, it's also suggested that OpenAI's newer models may have advanced beyond Ajax's capabilities as of September 2023.

Both The Information and analyst Jeff Pu claim that Apple will have some kind of generative AI feature available on the ‌iPhone‌ and iPad around late 2024, which is when iOS 18 will be coming out. Pu said in October that Apple is building a few hundred AI servers in 2023, with more to come in 2024. Apple will reportedly offer a combination of cloud-based AI and AI with on-device processing.

Tag: Apple GPT Guide

 

[bnew]

Apple wants AI to run directly on its hardware instead of in the cloud

iPhone maker wants to catch up to its rivals when it comes to AI.

arstechnica.com

Apple wants AI to run directly on its hardware instead of in the cloud​

iPhone maker wants to catch up to its rivals when it comes to AI.​

TIM BRADSHAW, FINANCIAL TIMES - 12/21/2023, 9:43 AM

Enlarge / The iPhone 15 Pro.

Apple

87​

Apple’s latest research about running large language models on smartphones offers the clearest signal yet that the iPhone maker plans to catch up with its Silicon Valley rivals in generative artificial intelligence.

The paper, entitled “LLM in a Flash,” offers a “solution to a current computational bottleneck,” its researchers write.

Its approach “paves the way for effective inference of LLMs on devices with limited memory,” they said. Inference refers to how large language models, the large data repositories that power apps like ChatGPT, respond to users’ queries. Chatbots and LLMs normally run in vast data centers with much greater computing power than an iPhone.

The paper was published on December 12 but caught wider attention after Hugging Face, a popular site for AI researchers to showcase their work, highlighted it late on Wednesday. It is the second Apple paper on generative AI this month and follows earlier moves to enable image-generating models such as Stable Diffusion to run on its custom chips.

Device manufacturers and chipmakers are hoping that new AI features will help revive the smartphone market, which has had its worst year in a decade, with shipments falling an estimated 5 percent, according to Counterpoint Research.

Despite launching one of the first virtual assistants, Siri, back in 2011, Apple has been largely left out of the wave of excitement about generative AI that has swept through Silicon Valley in the year since OpenAI launched its breakthrough chatbot ChatGPT. Apple has been viewed by many in the AI community as lagging behind its Big Tech rivals, despite hiring Google’s top AI executive, John Giannandrea, in 2018.

While Microsoft and Google have largely focused on delivering chatbots and other generative AI services over the Internet from their vast cloud computing platforms, Apple’s research suggests that it will instead focus on AI that can run directly on an iPhone.

Apple’s rivals, such as Samsung, are gearing up to launch a new kind of “AI smartphone” next year. Counterpoint estimated more than 100 million AI-focused smartphones would be shipped in 2024, with 40 percent of new devices offering such capabilities by 2027.

The head of the world’s largest mobile chipmaker, Qualcomm chief executive Cristiano Amon, forecast that bringing AI to smartphones would create a whole new experience for consumers and reverse declining mobile sales.

“You’re going to see devices launch in early 2024 with a number of generative AI use cases,” he told the Financial Times in a recent interview. “As those things get scaled up, they start to make a meaningful change in the user experience and enable new innovation which has the potential to create a new upgrade cycle in smartphones.”

More sophisticated virtual assistants will be able to anticipate users’ actions such as texting or scheduling a meeting, he said, while devices will also be capable of new kinds of photo editing techniques.

Google this month unveiled a version of its new Gemini LLM that will run “natively” on its Pixel smartphones.

Running the kind of large AI model that powers ChatGPT or Google’s Bard on a personal device brings formidable technical challenges, because smartphones lack the huge computing resources and energy available in a data center. Solving this problem could mean that AI assistants respond more quickly than they do from the cloud and even work offline.

Ensuring that queries are answered on an individual’s own device without sending data to the cloud is also likely to bring privacy benefits, a key differentiator for Apple in recent years.

“Our experiment is designed to optimize inference efficiency on personal devices,” its researchers said. Apple tested its approach on models including Falcon 7B, a smaller version of an open source LLM originally developed by the Technology Innovation Institute in Abu Dhabi.

Optimizing LLMs to run on battery-powered devices has been a growing focus for AI researchers. Academic papers are not a direct indicator of how Apple intends to add new features to its products, but they offer a rare glimpse into its secretive research labs and the company’s latest technical breakthroughs.

“Our work not only provides a solution to a current computational bottleneck but also sets a precedent for future research,” wrote Apple’s researchers in the conclusion to their paper. “We believe as LLMs continue to grow in size and complexity, approaches like this work will be essential for harnessing their full potential in a wide range of devices and applications.”

Apple did not immediately respond to a request for comment.

 

[bnew]

https://archive.is/9I7zW

https://archive.is/WUsng

 

[bnew]

Google DeepMind used a large language model to solve an unsolved math problem

They had to throw away most of what it produced but there was gold among the garbage.

www.technologyreview.com

ARTIFICIAL INTELLIGENCE

Google DeepMind used a large language model to solve an unsolved math problem​

They had to throw away most of what it produced but there was gold among the garbage.

By Will Douglas Heaven
archive page


December 14, 2023

STEPHANIE ARNETT/MITTR

Google DeepMind has used a large language model to crack a famous unsolved problem in pure mathematics. In a paper published in Nature today, the researchers say it is the first time a large language model has been used to discover a solution to a long-standing scientific puzzle—producing verifiable and valuable new information that did not previously exist. “It’s not in the training data—it wasn’t even known,” says coauthor Pushmeet Kohli, vice president of research at Google DeepMind.

Large language models have a reputation for making things up, not for providing new facts. Google DeepMind’s new tool, called FunSearch, could change that. It shows that they can indeed make discoveries—if they are coaxed just so, and if you throw out the majority of what they come up with.

FunSearch (so called because it searches for mathematical functions, not because it’s fun) continues a streak of discoveries in fundamental math and computer science that DeepMind has made using AI. FirstAlphaTensor found a way to speed up a calculation at the heart of many different kinds of code, beating a 50-year record. ThenAlphaDev found ways to make key algorithms used trillions of times a day run faster.

Yet those tools did not use large language models. Built on top of DeepMind’s game-playing AI AlphaZero, both solved math problems by treating them as if they were puzzles in Go or chess. The trouble is that they are stuck in their lanes, says Bernardino Romera-Paredes, a researcher at the company who worked on both AlphaTensor and FunSearch: “AlphaTensor is great at matrix multiplication, but basically nothing else.”

FunSearch takes a different tack. It combines a large language model called Codey, a version of Google’s PaLM 2 that isfine-tuned on computer code, with other systems that reject incorrect or nonsensical answers and plug good ones back in.

“To be very honest with you, we have hypotheses, but we don’t know exactly why this works,” saysAlhussein Fawzi, a research scientist at Google DeepMind. “In the beginning of the project, we didn’t know whether this would work at all.”

The researchers started by sketching out the problem they wanted to solve in Python, a popular programming language. But they left out the lines in the program that would specify how to solve it. That is where FunSearch comes in. It gets Codey to fill in the blanks—in effect, to suggest code that will solve the problem.

A second algorithm then checks and scores what Codey comes up with. The best suggestions—even if not yet correct—are saved and given back to Codey, which tries to complete the program again. “Many will be nonsensical, some will be sensible, and a few will be truly inspired,” says Kohli. “You take those truly inspired ones and you say, ‘Okay, take these ones and repeat.’”

After a couple of million suggestions and a few dozen repetitions of the overall process—which took a few days—FunSearch was able to come up with code that produced a correct and previously unknown solution to the cap set problem, which involves finding the largest size of a certain type of set. Imagine plotting dots on graph paper. The cap set problem is like trying to figure out how many dots you can put down without three of them ever forming a straight line.

It’s super niche, but important. Mathematicians do not even agree on how to solve it, let alone what the solution is. (It is also connected to matrix multiplication, the computation that AlphaTensor found a way to speed up.) Terence Tao at the University of California, Los Angeles, who has won many of the top awards in mathematics, including the Fields Medal, called the cap set problem “perhaps my favorite open question” in a 2007blog post.

Tao is intrigued by what FunSearch can do. “This is a promising paradigm,” he says. “It is an interesting way to leverage the power of large language models.”

A key advantage that FunSearch has over AlphaTensor is that it can, in theory, be used to find solutions to a wide range of problems. That’s because it produces code—a recipe for generating the solution, rather than the solution itself. Different code will solve different problems. FunSearch’s results are also easier to understand. A recipe is often clearer than the weird mathematical solution it produces, says Fawzi.

To test its versatility, the researchers used FunSearch to approach another hard problem in math: the bin packing problem, which involves trying to pack items into as few bins as possible. This is important for a range of applications in computer science, from data center management to e-commerce. FunSearch came up with a way to solve it that’s faster than human-devised ones.

Mathematicians are “still trying to figure out the best way to incorporate large language models into our research workflow in ways that harness their power while mitigating their drawbacks,” Tao says. “This certainly indicates one possible way forward.”

The headline of this article has been updated.

 

[bnew]

Google DeepMind used a large language model to solve an unsolved math problem

They had to throw away most of what it produced but there was gold among the garbage.

www.technologyreview.com

ARTIFICIAL INTELLIGENCE

This new system can teach a robot a simple household task within 20 minutes​

The Dobb-E domestic robotics system was trained in real people’s homes and could help solve the field’s data problem.

By

• Rhiannon Williams archive page

December 14, 2023

STEPHANIE ARNETT/MITTR | HELLO ROBOT, ENVATO

A new system that teaches robots a domestic task in around 20 minutes could help the field of robotics overcome one of its biggest challenges: a lack of training data.

The open-source system, called Dobb-E, was trained using data collected from real homes. It can help to teach a robot how to open an air fryer, close a door, or straighten a cushion, among other tasks.

While other types of AI, such as large language models, are trained on huge repositories of data scraped from the internet, the same can’t be done with robots, because the data needs to be physically collected. This makes it a lot harder to build and scale training databases.

Similarly, while it’s relatively easy to train robots to execute tasks inside a laboratory, these conditions don’t necessarily translate to the messy unpredictability of a real home.

To combat these problems, the team came up with a simple, easily replicable way to collect the data needed to train Dobb-E—using an iPhone attached to a reacher-grabber stick, the kind typically used to pick up trash. Then they set the iPhone to record videos of what was happening.

Volunteers in 22 homes in New York completed certain tasks using the stick, including opening and closing doors and drawers, turning lights on and off, and placing tissues in the trash. The iPhones’ lidar systems, motion sensors, and gyroscopes were used to record data on movement, depth, and rotation—important information when it comes to training a robot to replicate the actions on its own.

After they’d collected just 13 hours’ worth of recordings in total, the team used the data to train an AI model to instruct a robot in how to carry out the actions. The model used self-supervised learning techniques, which teach neural networks to spot patterns in data sets by themselves, without being guided by labeled examples.

The next step involved testing how reliably a commercially available robot called Stretch, which consists of a wheeled unit, a tall pole, and a retractable arm, was able to use the AI system to execute the tasks. An iPhone held in a 3D-printed mount was attached to Stretch’s arm to replicate the setup on the stick.

The researchers tested the robot in 10 homes in New York over 30 days, and it completed 109 household tasks with an overall success rate of 81%. Each task typically took Dobb-E around 20 minutes to learn: five minutes of demonstration from a human using the stick and attached iPhone, followed by 15 minutes of fine-tuning, when the system compared its previous training with the new demonstration.

Large language models combined with confidence scores help them recognize uncertainty. That could be key to making robots safe and trustworthy.

Once the fine-tuning was complete, the robot was able to complete simple tasks like pouring from a cup, opening blinds and shower curtains, or pulling board-game boxes from a shelf. It could also perform multiple actions in quick succession, such as placing a can in a recycling bag and then lifting the bag.

However, not every task was successful. The system was confused by reflective surfaces like mirrors. Also, because the robot’s center of gravity is low, tasks that require pulling something heavy at height, like opening fridge doors, proved too risky to attempt.

The research represents tangible progress for the home robotics field, says Charlie C. Kemp, cofounder of the robotics firm Hello Robot and a former associate professor at Georgia Tech. Although the Dobb-E team used Hello Robot’s research robot, Kemp was not involved in the project.

“The future of home robots is really coming. It’s not just some crazy dream anymore,” he says. “Scaling up data has always been a challenge in robotics, and this is a very creative, clever approach to that problem.”

To date, Roomba and other robotic vacuum cleaners are the only real commercial home robot successes, says Jiajun Wu, an assistant professor of computer science at Stanford University who was not involved in the research. Their job is easier because Roombas don’t interact with objects—in fact,a their aim is to avoid them. It’s much more challenging to develop home robots capable of doing a wider range of tasks, which is what this research could help advance.

The NYU research team has made all elements of the project open source, and they’re hoping others will download the code and help expand the range of tasks that robots running Dobb-E will be able to achieve.

“Our hope is that when we get more and more data, at some point when Dobb-E sees a new home, you don’t have to show it more examples,” says Lerrel Pinto, a computer science researcher at New York University who worked on the project.

“We want to get to the point when we don’t have to teach the robot new tasks, because it already knows all the tasks in most houses,” he says.

 

[bnew]

Allen & Overy rolls out AI contract negotiation tool in challenge to legal industry

Law firm works with Microsoft and AI start-up Harvey in attempt to ‘disrupt the legal market before someone disrupts us’

www.ft.com

Allen & Overy rolls out AI contract negotiation tool in challenge to legal industry​

Law firm works with Microsoft and AI start-up Harvey in attempt to ‘disrupt the legal market before someone disrupts us’

Allen & Overy has developed a service that draws on existing templates for contracts to draft new agreements that lawyers can then amend or accept © Robert Evans/Alamy

Cristina Criddle and Suzi Ring in London

YESTERDAY

177Print this page

Allen & Overy has created an artificial intelligence contract negotiation tool, as the magic circle law firm pushes forward with technology that threatens to disrupt the traditional practices of the legal profession.

The UK-headquartered group, in partnership with Microsoft and legal AI start-up Harvey, has developed the service which draws on existing templates for contracts, such as non-disclosure agreements and merger and acquisition terms, to draft new agreements that lawyers can then amend or accept.

The tool, known as ContractMatrix, is being rolled out to clients in an attempt to drive new revenues, attract more business and save time for in-house lawyers. A&O estimated it would save up to seven hours in contract negotiations.

More than 1,000 A&O lawyers are already using the tool, with five unnamed clients from banking, pharma, technology, media and private equity signed up to use the platform from January.

In a trial run, Dutch chipmaking equipment manufacturer ASML and health technology company Philips said they used the service to negotiate what they called the “world’s first 100 per cent AI generated contract between two companies”.

The legal sector is grappling with the rise of generative AI — technology that can review, extract and write large passages of humanlike text — which could result in losses of jobs and revenues by reducing billable hours and entry-level work for junior staff.

But David Wakeling, A&O partner and head of the firm’s markets innovation group, which developed ContractMatrix, said the firm’s goal was to “disrupt the legal market before someone disrupts us”.

“If we look at the impact of AI in law, it is happening and it’s likely to happen in a pretty big way in the next few years, and we are seeing it as an opportunity,” he added.

The firm is also planning to offer the service to the clients it gains from its $3.5bn merger with US outfit Shearman & Sterling, said Wakeling, which is due to complete by May.

The legal sector has been one of the fastest industries to adopt and experiment with generative AI technology off the back of the success of Microsoft-backed OpenAI’s ChatGPT and Google’s Bard. However, while some firms have heavily invested in new products, many are waiting to see what tools will win out.

A&O had previously introduced an AI-powered chatbot for use within the firm. Other law firms including Addleshaw Goddard, Travers Smith and Macfarlanes have also rolled out generative AI pilots internally.

A&O would not detail specific financial terms around the contract negotiation tool but said clients would pay an annual subscription fee per licence, with a minimum purchase of five users. The law firm is aiming to have subscriptions with hundreds of companies by the end of next year.

The product will also be available more widely to companies through Microsoft’s enterprise software marketplaces, Azure and AppSource, in the first half of 2024. Microsoft said the project would “deliver significant value to [A&O] customers”.

Concerns have been raised around using generative AI in the legal sector because of issues related to data privacy and client confidentiality, as well as so-called hallucinations, where a model generates incorrect information.

Wakeling said that hallucinations could happen with ContractMatrix, but they were significantly reduced because of the templates its underlying technology had been trained on. He added that client data was also not used to train the AI models that underpin the software, and inputs and outputs are encrypted.

However, if clients wished to make the tool more effective and personalised, A&O said it could work with a business to fine-tune the model and train on their data.

“We are seeing it as a big open market opportunity . . . because in-house lawyers need efficiency and productivity gains as well,” Wakeling added. “They can be that much quicker and that much more efficient than their competitors. And you would expect that to be attractive to clients because it’s generally a bit cheaper, a bit faster, a bit better.”

Additional reporting by Tim Bradshaw

 

[bnew]

Apple Explores A.I. Deals With News Publishers

The company has discussed multiyear deals worth at least $50 million to train its generative A.I. systems on publishers’ news articles.

www.nytimes.com

Apple Explores A.I. Deals With News Publishers​

The company has discussed multiyear deals worth at least $50 million to train its generative A.I. systems on publishers’ news articles.

The negotiations mark one of the earliest examples of how Apple is trying to catch up to rivals in the race to develop generative A.I.Credit...Karsten Moran for The New York Times

By Benjamin Mullin and Tripp Mickle

Benjamin Mullin covers the companies behind news and entertainment from New York. Tripp Mickle covers Apple from San Francisco.

Dec. 22, 2023

Apple has opened negotiations in recent weeks with major news and publishing organizations, seeking permission to use their material in the company’s development of generative artificial intelligence systems, according to four people familiar with the discussions.

The technology giant has floated multiyear deals worth at least $50 million to license the archives of news articles, said the people with knowledge of talks, who spoke on the condition of anonymity to discuss sensitive negotiations. The news organizations contacted by Apple include Condé Nast, publisher of Vogue and The New Yorker; NBC News; and IAC, which owns People, The Daily Beast and Better Homes and Gardens.

The negotiations mark one of the earliest examples of how Apple is trying to catch up to rivals in the race to develop generative A.I., which allows computers to create images and chat like a human. The technology, which artificial intelligence experts refer to as neural networks, is built by using troves of photos or digital text to recognize patterns. By analyzing thousands of cat photos, for instance, a computer can learn to recognize a cat.

Microsoft, OpenAI, Google, Meta and other companies have released chatbots and other products built with the technology. The tools could change the way people work and generate billions of dollars in sales.

But Apple has been absent from the public discussion of A.I. Its virtual assistant, Siri, has remained largely stagnant in the decade since its release.

A spokeswoman for Apple declined to comment. During a call with analysts last month, Tim Cook, the company’s chief executive, said Apple has work “going on” connected to A.I. but declined to elaborate.

Some of the publishers contacted by Apple were lukewarm on the overture. After years of on-again-off-again commercial deals with tech companies like Meta, the owner of Facebook, publishers have grown wary of jumping into business with Silicon Valley.

Several publishing executives were concerned that Apple’s terms were too expansive, according to three people familiar with the negotiations. The initial pitch covered broad licensing of publishers’ archives of published content, with publishers potentially on the hook for any legal liabilities that could stem from Apple’s use of their content.

Apple was also vague about how it intended to apply generative A.I. to the news industry, the people said, a potential competitive risk given Apple’s substantial audience for news on its devices.

Still, some news executives were optimistic that Apple’s approach might eventually lead to a meaningful partnership. Two people familiar with the discussions struck a positive note on the long-term prospects of a deal, contrasting Apple’s approach of asking for permission with behavior from other artificial intelligence-enabled companies, which have been accused of seeking licensing deals with news organizations after they had already used their content to train its generative models.

In recent years, Apple executives have been debating how to accumulate the data needed to build generative A.I. products, according to two people familiar with the work. Some of its rivals have been accused of taking written material from across the internet without the permission of the artists, writers and coders who created it, leading to several copyright lawsuits.

Apple has been reluctant to take information from the internet, partly because of its commitment to privacy. After it acquired the social analytics start-up Topsy in 2013, Apple’s leadership asked that Topsy stop collecting information from Twitter, saying that doing so violated the company’s policy against collecting data on Apple customers, who might also post on the social media site, these two people said.

The explosion of artificial intelligence has raised alarms among news executives, many of whom are concerned that generative A.I. products like OpenAI’s ChatGPT could draw in readers who would otherwise consume their news on platforms for their own subscribers and advertisers.

Print news organizations, which decades ago saw their lucrative classifieds business demolished by online competitors, have been particularly wary about striking deals with A.I. organizations, engaging cautiously with an eye toward preserving their existing businesses.

In a statement, an OpenAI spokesman said that the company respects “the rights of content creators and owners and believes they should benefit from A.I. technology,” citing its recent deals with the American Journalism Project and the German publisher Axel Springer.

“We’re optimistic we will continue to find mutually beneficial ways to work together in support of a rich news ecosystem,” the OpenAI spokesman said.

 

[bnew]

TomTom and Microsoft team up to bring generative AI to automobiles

TomTom is releasing a generative AI platform for automobiles which was developed with the help of Microsoft’s Azure OpenAI service. You’ll be able to navigate, set up infotainment and a whole lot more just by conversing with the bot.

www.engadget.com

TomTom and Microsoft team up to bring generative AI to automobiles​

Get ready for a ‘fully integrated’ conversational driving assistant.​

Lawrence Bonk

Contributing Reporter

Updated Tue, Dec 19, 2023, 1:30 AM EST·1 min read

TomTom

TomTom just announced a “fully integrated, AI-powered conversational automotive assistant” which should start popping up in dashboard infotainment platforms in the near-ish future. The company has issued some bold claims for the AI, saying it’ll offer “more sophisticated voice interaction” and allow users to converse naturally to navigate, find stops along a route, control onboard systems, open windows and just about anything else you find yourself doing while driving.

The company, best known for GPS platforms, partnered up with Microsoft to develop this AI assistant. The technology leverages OpenAI’s large language models, in addition to Microsoft products like Azure Cosmos DB and Azure Cognitive Services. Cosmos DB is a multi-model database and Cognitive Services is a set of APIs for use in AI applications, so this should be a capable assistant that draws from the latest advancements.

TomTom promises that the voice assistant will integrate into a variety of interfaces offered by major automobile manufacturers, stating that the auto company will retain ownership of its branding. So this could start showing up in cars from a wide variety of makers. The company hasn’t announced any definitive partnerships with known vehicle manufacturers, but the technology will be integrated into TomTom’s proprietary Digital Cockpit, an open and modular in-vehicle infotainment platform.

This isn’t the first time a company has tried to stuff an LLM inside of a car. Back in June, Mercedes announced a three-month beta program that incorporated ChatGPT models into select vehicles. This tool also leveraged Microsoft’s Azure OpenAI service. TomTom is showing off the AI at CES in January, so we’ll know more about how it actually works at that point.

 

[bnew]

GitHub - apple/ml-ferret

Contribute to apple/ml-ferret development by creating an account on GitHub.

github.com

Ferret: Refer and Ground Anything Anywhere at Any Granularity​

An End-to-End MLLM that Accept Any-Form Referring and Ground Anything in Response. [Paper]

Haoxuan You*, Haotian Zhang*, Zhe Gan, Xianzhi Du, Bowen Zhang, Zirui Wang, Liangliang Cao, Shih-Fu Chang, Yinfei Yang [*: equal contribution]

Overview​

Diagram of Ferret Model.​

Key Contributions:

• Ferret Model - Hybrid Region Representation + Spatial-aware Visual Sampler enable fine-grained and open-vocabulary referring and grounding in MLLM.
• GRIT Dataset (~1.1M) - A Large-scale, Hierarchical, Robust ground-and-refer instruction tuning dataset.
• Ferret-Bench - A multimodal evaluation benchmark that jointly requires Referring/Grounding, Semantics, Knowledge, and Reasoning.

 

[bnew]

[2312.13558] The Truth is in There: Improving Reasoning in Language Models with Layer-Selective Rank Reduction

Computer Science > Machine Learning​

[Submitted on 21 Dec 2023]

The Truth is in There: Improving Reasoning in Language Models with Layer-Selective Rank Reduction​

Pratyusha Sharma, Jordan T. Ash, Dipendra Misra

Transformer-based Large Language Models (LLMs) have become a fixture in modern machine learning. Correspondingly, significant resources are allocated towards research that aims to further advance this technology, typically resulting in models of increasing size that are trained on increasing amounts of data. This work, however, demonstrates the surprising result that it is often possible to significantly improve the performance of LLMs by selectively removing higher-order components of their weight matrices. This simple intervention, which we call LAyer-SElective Rank reduction (LASER), can be done on a model after training has completed, and requires no additional parameters or data. We show extensive experiments demonstrating the generality of this finding across language models and datasets, and provide in-depth analyses offering insights into both when LASER is effective and the mechanism by which it operates.

Subjects:	Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV)
Cite as:	arXiv:2312.13558 [cs.LG]
	(or arXiv:2312.13558v1 [cs.LG] for this version)
	[2312.13558] The Truth is in There: Improving Reasoning in Language Models with Layer-Selective Rank Reduction Focus to learn more

Submission history​

From: Dipendra Misra [view email]
[v1] Thu, 21 Dec 2023 03:51:08 UTC (3,576 KB)

https://arxiv.org/pdf/2312.13558.pdf

 

[bnew]

Apple quietly released an open source multimodal LLM in October

Apple is belatedly being celebrated on social media for releasing Ferret, an open source multimodal LLM, in October 2023.

venturebeat.com

Apple quietly released an open source multimodal LLM in October​

Sharon Goldman @sharongoldman

December 23, 2023 6:44 AM

Image created by DALL-E 3 for VentureBeat

Are you ready to bring more awareness to your brand? Consider becoming a sponsor for The AI Impact Tour. Learn more about the opportunities here.

---

With little fanfare, researchers from Apple and Columbia University released an open source multimodal LLM, called Ferret, in October 2023. At the time, the release — which included the code and weights, but for research use only, not a commercial license — did not receive much attention. But now that may be changing: With open source models from Mistral making recent headlines and Google’s Gemini model is coming to the Pixel Pro and eventually to Android, there has been increased chatter about the potential for local LLMs to power small devices.

That chatter increased recently because Apple announced it had made a key breakthrough in deploying LLMs on iPhones: The company released two new research papers introducing new techniques for 3D avatars and efficient language model inference. The advancements were hailed as potentially enabling more immersive visual experiences and allowing complex AI systems to run on consumer devices such as the iPhone and iPad.

Many in the AI community who belatedly noticed the Ferret release celebrated Apple’s unexpected entry into the open source LLM landscape, especially since Apple has traditionally been known as a “walled garden.”

This morning, Bart de Witte, who runs a European non-profit focused on open source AI in medicine, posted on X: “I somehow missed this,” he wrote. “Apple joined the open source AI community in October. Ferret’s introduction is a testament to Apple’s commitment to impactful AI research, solidifying its place as a leader in the multimodal AI space…ps: I’m looking forward to the day when Local Large Language Models (LLLMs) run on my iPhone as an integrated service of a re-designed iOS.”

 

[bnew]

https://archive.is/5cN2T

Ferret: A Multimodal Large Language Model | Hacker News

news.ycombinator.com

---

Apple's 'Ferret' is a new open-source machine learning model​

Malcolm Owen | Dec 24, 2023​

A ferret in the wild [Pixabay/Michael Sehlmeyer]


​

Researchers working for Apple and from Cornell University quietly pushed an open-source multimodal LLM in October, a research release called "Ferret" that can use regions of images for queries.​

The introduction in October to Github largely flew under the radar, with no announcement or fanfare for its introduction. The code for Ferret was released alongside Ferret-Bench on October 30, with checkpoint releases introduced on December 14.

While it didn't receive much attention at first, the release became more of a big deal to AI researchers on Saturday, reports VentureBeat. Bart De Witte, operator of an AI-in-medicine non-profit, posted to X about the "missed" release, calling it a "testament to Apple's commitment to impactful AI research."

Ferret's release to open-source is being performed under a non-commercial license, so it cannot be commercialized in its current state. However, there's always a possibility for it to become used in a future Apple product or service in some way.

A tweet from October by Apple AI/ML research scientist Zhe Gan explains Ferret's use as being a system that can "refer and ground anything anywhere at any granularity" in an image. It can also do so by using any shape of region within an image.

In simpler terms, the model can examine a region drawn on an image, determine the elements within it that are of use to a user in a query, identify it, and draw a bounding box around the detected element. It can then use that identified element as part of a query, which it can then respond to in a typical fashion.

For example, highlighting an image of an animal in an image and asking the LLM what the animal is, it could determine the creature's species and that the user is referring to an individual animal from a group. It could then use the context of other items detected in the image to offer up further responses.



The release is important to researchers, as it shows Apple is keen to be more open with its AI work, rather than its usual secretive stance.

There's also the problem of infrastructure for Apple, as while it is working to increase the number of AI servers it owns, it may not have the scale available at the moment to work toe-to-toe with ChatGPT, for example. Though Apple could work with other firms to scale its capabilities, the other route is to do what it has just done, namely release an open-source model.

In one interesting element from the Github release, Reddit's r/Apple spotted that Ferret is "trained on 8 A100 GPUs with 80GB memory." Given Apple's history with Nvidia GPU support, this was seen to be a rare acknowledgment of the GPU producer.​

 

[bnew]

https://archive.is/UWVrE

[2311.05772] ADaPT: As-Needed Decomposition and Planning with Language Models

Computer Science > Artificial Intelligence​

[Submitted on 8 Nov 2023]

ADaPT: As-Needed Decomposition and Planning with Language Models​

Archiki Prasad, Alexander Koller, Mareike Hartmann, Peter Clark, Ashish Sabharwal, Mohit Bansal, Tushar Khot

Large Language Models (LLMs) are increasingly being used for interactive decision-making tasks requiring planning and adapting to the environment. Recent works employ LLMs-as-agents in broadly two ways: iteratively determining the next action (iterative executors) or generating plans and executing sub-tasks using LLMs (plan-and-execute). However, these methods struggle with task complexity, as the inability to execute any sub-task may lead to task failure. To address these shortcomings, we introduce As-Needed Decomposition and Planning for complex Tasks (ADaPT), an approach that explicitly plans and decomposes complex sub-tasks as-needed, i.e., when the LLM is unable to execute them. ADaPT recursively decomposes sub-tasks to adapt to both task complexity and LLM capability. Our results demonstrate that ADaPT substantially outperforms established strong baselines, achieving success rates up to 28.3% higher in ALFWorld, 27% in WebShop, and 33% in TextCraft -- a novel compositional dataset that we introduce. Through extensive analysis, we illustrate the importance of multilevel decomposition and establish that ADaPT dynamically adjusts to the capabilities of the executor LLM as well as to task complexity.

Comments:	Project Page: this https URL
Subjects:	Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG)
Cite as:	arXiv:2311.05772 [cs.AI]
	(or arXiv:2311.05772v1 [cs.AI] for this version)
	[2311.05772] ADaPT: As-Needed Decomposition and Planning with Language Models Focus to learn more

Submission history​

From: Archiki Prasad [view email]

[v1] Wed, 8 Nov 2023 17:59:15 UTC (8,543 KB)

https://arxiv.org/pdf/2311.05772.pdf