The A.I Megathread (LLM , GPT , Development)

[bnew]

Harvard Business Publishing Education

hbsp.harvard.edu

AN INSPIRING MINDS SERIES

Student Use Cases for AI​

Start by Sharing These Guidelines with Your Class
by Ethan Mollick and Lilach Mollick

September 25, 2023


Getty Images / Shutterstock / HBP Staff

Explore more​

• CLASSROOM MANAGEMENT
• DIGITAL LEARNING
• TECHNOLOGY

Share this article

PrintEmailTweetSharePost

Generative AI tools and the large language models (LLMs) they’re built on create exciting opportunities and pose enormous challenges for teaching and learning. After all, AI can now be ubiquitous in the classroom; every student and educator with a computer and internet has free access to the most powerful AI models in the world. And, like any tool, AI offers both new capabilities and new risks.

To help you explore some of the ways students can use this disruptive new technology to improve their learning—while making your job easier and more effective—we’ve written a series of articles that examine the following student use cases:

1. AI as feedback generator
2. AI as personal tutor
3. AI as team coach
4. AI as learner

For each of these roles, we offer practical recommendations—and a detailed, shareable prompt—for how exactly you can guide students in wielding AI to achieve these ends.

But before you assign or encourage students to use AI, it’s important to first establish some guidelines around properly using these tools. That way, there’s less ambiguity about what students can expect from the AI, from “hallucinations” to privacy concerns.

Since these guidelines can be used generally—and across all four use cases we propose in this series—we wanted to share them in this introductory article. These are the same guidelines we provide our own students; feel free to use or adapt them for your class.

Student guidelines for proper AI use​

Understanding LLMs​

LLMs are trained on vast amounts of content that allows them to predict what word should come next in written text, much like the autocomplete feature in search bars. When you type something (called a prompt) into ChatGPT or another LLM, it tries to extend the prompt logically based on its training. Since LLMs like ChatGPT have been pre-trained on large amounts of information, they’re capable of many tasks across many fields. However, there is no instruction manual that comes with LLMs, so it can be hard to know what tasks they are good or bad at without considerable experience. Keep in mind that LLMs don’t have real understanding and often make mistakes, so it’s up to the user to verify their outputs.

Benefits and challenges of working with LLMs​

• Fabrication. AI can lie and produce plausible-sounding but incorrect information. Don’t trust anything it says at face value. If it gives you a number or fact, assume it is wrong unless you either know the answer or can check with another source. You will be responsible for any errors or omissions provided by the tool. It works best for topics you understand and can verify. Larger LLMs (like GPT-4) fabricate less, but all AIs fabricate to some degree.
• AI bias. AI can carry biases, stemming from its training data or human intervention. These biases vary across LLMs and can range from gender and racial biases to biases against particular viewpoints, approaches, or political affiliations. Each LLM has the potential for its own set of biases, and those biases can be subtle. You will need to critically consider answers and be aware of the potential for these sorts of biases.
• Privacy concerns. When data is entered into the AI, it can be used for future training. While ChatGPT offers a privacy mode that claims not to use input there for future AI training, the current state of privacy remains unclear for many models, and the legal implications are often uncertain. Do not share anything with AI that you want to keep private.

Best practices for AI interactions​

When interacting with AI, remember the following:

• You are accountable for your own work. Take every piece of advice or explanation given by AI critically and evaluate that advice independently.
• AI is not a person, but it can act like one. It’s very easy to read human intent into AI responses, but AI is not a real person responding to you. It is capable of a lot, but it doesn’t know you or your context. It can also get stuck in a loop, repeating similar content over and over.

“AI can now be ubiquitous in the classroom; every student and educator with a computer and internet has free access to the most powerful AI models in the world.”

• AI is unpredictable. AI has trained on billions of documents on the web, and it tries to fulfill or respond to your prompt reasonably based on what it has read. But you can’t know ahead of time what it’s going to say. The very same prompt can get a radically different response from the AI each time you use it. That means that your classmates may get different responses, as will trying the prompt more than once yourself.
• You are in charge. If the AI gets stuck in a loop and you’re ready to move on, then direct the AI to do what you’d like.
• Only share what you are comfortable sharing. Do not feel compelled to share anything personal, even if the AI asks. Anything you share may be used as training data for the AI.
• Try another LLM. If the prompt doesn’t work in one LLM, try another. Remember that an AI’s output isn’t consistent and will vary. Take notes and share what worked for you.

To communicate more effectively with AI:

• Seek clarity. If something isn’t clear, don’t hesitate to ask the AI to expand its explanation or give you different examples. If you are confused by the AI’s output, ask it to use different wording. You can keep asking until you get what you need. Interact with it naturally, asking questions and pushing back on its answers.
• Provide context. The AI can provide better help if it knows where you’re having trouble. The more context you give it, the more likely it is to be useful to you. It often helps to give the AI a role: “You are a friendly teacher who explains economics concepts to college students in introductory courses,” for example.
• Don’t assume the AI is tracking the conversation. LLMs have limited memory; if it seems to be losing track, remind it of what you need and keep asking it questions.

Preparing students to work more effectively with AI​

These guidelines help clarify what LLMs are and what students need to know to productively work with these tools. If you choose to share these guidelines, or a version of them, your students will have a better understanding of what to expect when interacting with AI and how to communicate their needs more effectively.

STUDENT USE CASES FOR AI: AN INSPIRING MINDS SERIES​

Prologue: Student Guidelines for AI Use
Part 1: AI as Feedback Generator
Part 2: AI as Personal Tutor
Part 3: AI as Team Coach
Part 4: AI as Learner


Now, you’re ready to explore the rest of our series on student uses for AI beginning with “Part 1: AI as Feedback Generator,” which tackles one of educators’ most laborious tasks: giving frequent feedback to students.

From the editors: As you read this series, share with us how you are using generative AI in your classes. What is your experience so far? What are your biggest concerns? What use cases have you found beneficial? We look forward to learning from you.

 

[bnew]

Why Open Source AI Will Win

You shouldn't bet against the bazaar or the GPU p̶o̶o̶r̶ hungry.

varunshenoy.substack.com

Why Open Source AI Will Win​

You shouldn't bet against the bazaar or the GPU p̶o̶o̶r̶ hungry.​

VARUN

SEP 15, 2023

Linux is subversive. Who would have thought even five years ago (1991) that a world-class operating system could coalesce as if by magic out of part-time hacking by several thousand developers scattered all over the planet, connected only by the tenuous strands of the Internet?

Certainly not I.

opening remarks in The Cathedral and the Bazaar by Eric Raymond.

---

There’s a popular floating theory on the Internet that a combination of the existing foundation model companies will be the end game for AI.

In the near future, every company will rent a “brain” from a model provider, such as OpenAI/Anthropic, and build applications that build on top of its cognitive capabilities.

In other words, AI is shaping up to be an oligopoly of sorts, with only a small set of serious large language model (LLM) providers.

I don’t think this could be farther from the truth. I truly believe that open source will have more of an impact on the future of LLMs and image models than the broad public believes.

There are a few arguments against open source that I see time and time again.

1. Open source AI cannot compete with the resources at industry labs. Building foundation models is expensive, and non-AI companies looking to build AI features will outsource their intelligence layer to a company that specializes in it. Your average company cannot scale LLMs or produce novel results the same way a well capitalized team of talented researchers can. On the image generation side, Midjourney is miles ahead of anything else.
2. Open source AI is not safe. Mad scientists cooking up intelligence on their cinderblock-encased GPUs will not align their models with general human interests
   1
   .
   
   
3. Open source AI is incapable of reasoning. Not only do open source models perform more poorly than closed models on benchmarks, but they also lack emergent capabilities, those that would enable agentic workflows, for example.

While they seem reasonable, I think these arguments hold very little water.

LLMs are business critical​

​

Outsourcing a task is fine — when the task is not business critical.
Infrastructure products save users from wasting money and energy on learning Kubernetes or hiring a team of DevOps engineers. No company should have to hand-roll their own HR/bill payments software. There are categories of products that enable companies to “focus on what makes their beer taste better”
2
.


LLMs, for the most part, do not belong in this category. There are some incumbents building AI features on existing products, where querying OpenAI saves them on hiring ML engineers. For them, leveraging closed AI makes sense.

However, there’s a whole new category of AI native businesses for whom this risk is too great. Do you really want to outsource your core business, one that relies on confidential data, to OpenAI or Anthropic? Do you want to spend the next few years of your life working on a “GPT wrapper”?

Obviously not.

If you’re building an AI native product, your primary goal is getting off of OpenAI as soon as you possibly can. Ideally, you can bootstrap your intelligence layer using a closed source provider, build a data flywheel from engaged users, and then fine-tune your own models to perform your tasks with higher accuracy, less latency, and more control.

Every business needs to own their core product, and for AI native startups, their core product is a model trained on proprietary data
3
. Using closed source model providers for the long haul exposes an AI native company to undue risk.


There is too much pressure pent up for open source LLMs to flop. The lives of many companies are at stake. Even Google has acknowledged that they have no moat in this new world of open source AI.

Reasoning doesn’t actually matter​

​

The general capabilities of LLMs open them up to an exponential distribution of use cases. The most important tasks are fairly straightforward: summarization, explain like I’m 5, create a list (or some other structure) from a blob of text, etc.

Reasoning, the type you get from scaling these models to get larger, doesn’t matter for 85% of use cases. Researchers love sharing that their 200B param model can solve challenging math problems or build a website from a napkin sketch, but I don’t think most users (or developers) have a burning need for these capabilities.

The truth is that open source models are incredibly good at the most valuable tasks, and can be fine-tuned to cover likely up to 99% of use-cases when a product has collected enough labeled data.


Fine-tuned Llama 2 models vs. GPT-4 (from Anyscale)

Reasoning, the holy grail that researchers are chasing, probably doesn’t matter nearly as much as people think.

More important than reasoning is context length and truthfulness.

Let’s start with context length. The longer the context length for a language model, the longer the prompts and chat logs you can pass in.

The original Llama has a context length of 2k tokens. Llama 2 has a context length of 4k.

Earlier this year, an indie AI hacker discovered that a single line code change to the RoPE embeddings for Llama 2 would give you up to 8K of context length for free with no additional training.

Just last week another indie research project was released, YaRN, that extends Llama 2’s context length to 128k tokens.

I still don’t have access to GPT-4 32k. This is the speed of open source.


While contexts have scaled up, the hardware requirements to run massive models have also scaled down. You can now run state-of-the-art massive language models from your Macbook thanks to projects like Llama.cpp. Being able to use these models locally is a huge plus for security and costs as well. In the limit, you can run your models on your users’ hardware. Models are continuing to scale down while retaining quality. Microsoft’s Phi-1.5 is only 1.3 billion parameters but meets Llama 2 7B on several benchmarks. Open source LLM experimentation will continue to explode as consumer hardware and the GPU poor rise to the challenge.

On truthfulness: out-of-the-box open source models are less truthful than closed source models, and I think this is actually fine. In many cases, hallucination can be a feature, not a bug, particularly when it comes to creative tasks like storytelling.

Closed AI models have a certain filter that make them sound artificial and less interesting. MythoMax-L2 tells significantly better stories than Claude 2 or ChatGPT, at only 13B parameters. When it comes to honestly, the latest open source LLMs work well with retrieval augmented generation, and they will only get better.

Control above all else​

​

Let’s take a brief look at the image generation side.

I would argue that Stable Diffusion XL (SDXL), the best open source model, is nearly on-par with Midjourney.


Stable Diffusion XL generations for the prompt “an astronaut playing a guitar on Mars with a llama”. These images were generated on the first try, no cherry-picking needed.

In exchange for the slightly worse ergonomics, Stable Diffusion users have access to hundreds of community crafted LoRAs
4
, fine-tunes, and textual embeddings. Users quickly discovered hands were a sore for SDXL, and within weeks a LoRA that fixes hands appeared online.

Other open source projects like ControlNet give Stable Diffusion users significantly more power when it comes to structuring their outputs, where Midjourney falls flat.


A flowchart of how Stable Diffusion + ControlNet works. Clipped from here.

Moreover, Midjourney doesn’t have an API, so if you want to build a product with an image diffusion feature, you would have to use Stable Diffusion in some form.


This image went viral on Twitter and Reddit this week. It uses Stable Diffusion with ControlNet. Currently, you can’t create images like this on Midjourney.

 

[bnew]

{continued}

There are similar controllable features and optimizations that open source LLMs enable.

An LLM’s logits, the token-wise probability mass function at each iteration, can be used to generate structured output. In other words, you can guarantee the generation of JSON without entering a potentially expensive “validate-retry” loop, which is what you would need to do if you were using OpenAI.


An example of logits from NVIDIA.

Open source models are smaller and run on your own dedicated instance, leading to lower end-to-end latencies. You can improve throughput by batching queries and using inference servers like vLLM.

There are many more tricks (see: speculative sampling, concurrent model execution, KV caching) that you can apply to improve on the axes of latency and throughput. The latency you see on the OpenAI endpoint is the best you can do with closed models, rendering it useless for many latency-sensitive products and too costly for large consumer products.

Thanks for reading Public Experiments! Subscribe to be notified when I post next.

On top of all this, you can also fine-tune or train your own LoRAs on top of open source models with maximal control. Frameworks like Axolotl and TRL have made this process simple
5
. While closed source model providers also have their own fine-tuning endpoints, you wouldn’t get the same level of control or visibility than if you did it yourself.



Falcon 180B, the largest open source model to date, was released last week. Within hours, Discords filled with mostly anonymous developers began exploring how they could recreate GPT-4 using this new model as a base layer.

Open source also provides guarantees on privacy and security.

You control the inflow and outflow of data in open models. The option to self-host is a necessity for many users, especially those working in regulated fields like healthcare. Many applications will also need to run on proprietary data, on both the training and inference side.

Security is best explained by Linus’s Law:

Given a large enough beta-tester and co-developer base, almost every problem will be characterized quickly and the fix obvious to someone.

Or, less formally, ‘‘Given enough eyeballs, all bugs are shallow.’’

Linux succeeded because it was built in the open. Users knew exactly what they were getting and had the opportunity to file bugs or even attempt to fix them on their own with community support.

The same is true for open source models. Even software 2.0 needs to be audited. Otherwise, things can change under the hood, leading to regressions in your application. This is unacceptable for most business use cases.

This paper recently showed that OpenAI’s endpoints drift over time. You cannot be confident that a prompt that works flawlessly will perform the same a month from now.

Adopting an open source approach for AI technology can create a wide-reaching network of checks and balances. Scientists and developers globally can peer-review, critique, study, and understand the underlying mechanisms, leading to improved safety, reliability, interpretability, and trust. Furthermore, widespread knowledge helps advance the technology responsibly while mitigating the risk of its misuse. Hugging Face is the new RedHat.

You can only trust models that you own and control. The same can’t be said for black box APIs. This is also why the AI safety argument against open source makes zero sense. History suggests, open source AI is, in fact, safer.

The Real Problem is Hype​

​

Why do people currently prefer closed source? Two reasons: ease-of-use and mindshare.

Open source is much harder to use than closed source models. It seems like you need to hire a team of machine learning engineers to build on top of open source as opposed to using the OpenAI API. This is ok, and will be true in the short-term. This is the cost of control and the rapid pace of innovation. People who are willing to spend time at the frontier will be treated by being able to build much better products. The ergonomics will get better.

The more unfortunate issue is mindshare.

Closed source model providers have captured the collective mindshare of this AI hype cycle. People don’t have time to mess around with open source nor do they have the awareness of what open source is capable of. But they do know about OpenAI, Pinecone, and LangChain.

The “OPL” stack, from Wen Yang.

Using the right tool is often conflated with using the best known tool. The current hype cycle has put closed source AI in the spotlight. As open source offerings mature and become more user-friendly and customizable, they will emerge as the superior choice for many applications.

Rather than getting swept up in the hype, forward-thinking organizations will use this period to deeply understand their needs and lay the groundwork to take full advantage of open source AI. They will build defensible and differentiated AI experiences on open technology. This measured approach enables a sustainable competitive advantage in the long run.

The future remains bright for pragmatic adopters who see past the hype and keep their eyes on the true prize: truly open AI.
1

Side note: alignment might hurt overall performance, according to this recently published paper.
2

From Jeff Bezos’ talk at YC ‘08.
3

In many ways, these models are just reflections of their underlying training data. In fact, model size doesn’t matter nearly as much. A 7B open-source model fine-tuned on SQL queries will outperform GPT-4.
4

Short for “low-rank approximation”, a technique used to train a small set of model weights (called an adapter) that can then be merged into the main model weights. It’s a more light-weight approach to fine-tuning.


When training a LoRA, the pretrained weights for a model are untouched. Only the A and B matrices above are trained. The final trained adapter (usually only a few MB in size) can then be used on top of pretrained models or merged with the pretrained weights and offered as a new model altogether. Diagram from the original LoRA paper.

5

Similar to how DevOps relies on Infrastructure as Code (IaC) principles, Axolotl enables engineers to detail fine-tuning processes using YAML. As fine-tuning standardizes further, its accessibility improves even for those without deep ML experience.

 

[Micky Mikey]

Where do guys see the field of A.I. being a year from now?

AGI?
Opensource LLM models being more capable than ChatGPT 4?
A.I. generated tv shows?
Google dethroning OpenAI?
Autonomous A.I. agents getting more powerful?

 

[bnew]

Where do guys see the field of A.I. being a year from now?

AGI?
Opensource LLM models being more capable than ChatGPT 4?
A.I. generated tv shows?
Google dethroning OpenAI?
Autonomous A.I. agents getting more powerful?

I think a year from now someone will have figured out how to get LLM's running using even lower hardware requirements, which will even make it cheaper to offer and widen adoption.

 

[bnew]

 

[bnew]

some really cool requests for DALL-E 3 images

 

[bnew]

Meta is putting AI chatbots everywhere

The Meta AI assistant is coming to WhatsApp, Messenger, and Instagram, along with dozens of AI characters based on celebrities like MrBeast and Charli D’Amelio.

www.theverge.com

• META

Meta is putting AI chatbots everywhere​

The Meta AI assistant is coming to WhatsApp, Messenger, and Instagram, along with dozens of AI characters based on celebrities like MrBeast and Charli D’Amelio.​

By Alex Heath, a deputy editor and author of the Command Line newsletter. He’s covered the tech industry for over a decade at The Information and other outlets.


Sep 27, 2023, 5:30 PM UTC

An example of what Meta’s AI assistant can do. Meta

Meta is officially entering the AI chatbot wars, starting with its own assistant and a slew of AI characters it’s releasing in WhatsApp, Instagram, and Messenger.

For anyone who has used OpenAI’s ChatGPT, or other chatbots like Anthropic’s Claude, Meta’s AI will immediately feel familiar. Meta sees it as a general-purpose assistant for everything from planning a trip with friends in a group chat to answering questions you’d normally ask a search engine. On that latter piece, Meta is announcing a partnership with Microsoft’s Bing to provide real-time web results, which sets Meta AI apart from a lot of the other free AIs out there that don’t have super recent information.

Another big aspect of the Meta AI is its ability to generate images like Midjourney or OpenAI’s DALL-E via the prompt “/imagine.” In my brief demo, it produced compelling high-res photos in a few seconds. Like all of Meta’s AI features being announced this week, this image generation is totally free to use.

Ahmad Al-Dahle, Meta’s VP of generative AI who has been leading the assistant’s development, wouldn’t tell me exactly what it’s trained on. He described it as a “custom-made” large language model that is “based on a lot of the core principles behind Llama 2,” Meta’s latest quasi-open source model that is being quickly adopted across various industries.

The rapid adoption of Llama 2 has helped Meta refine how its own assistant works, he says. “We just saw huge demand for the models, and then we saw an incredible amount of innovation happening on the models that really helped us understand their performance, understand their weaknesses, and help us iterate and leverage some of those components directly into product.”

In terms of how Meta AI differs from Llama 2, Al-Dahle says his team spent time “refining additional data sets for conversations so that we can create a tone that is conversational and friendly in the way that the assistant responds. A lot of existing AIs can be like robotic or bland.” Meta expanded the model’s context window, or the ability to leverage previous interactions to generate what the model produces next, “so that we can build a deeper, more capable back and forth” with users. He says Meta AI has also been tuned to give “very concise” answers.

Some of Meta’s AI characters are familiar faces. Image: Meta

Alongside Meta’s assistant, the company is beginning to roll out an initial roster of 28 AI characters across its messaging apps. Many of them are based on celebrities like Charli D’Amelio, Dwyane Wade, Kendall Jenner, MrBeast, Snoop Dogg, and Paris Hilton. Others are themed to specific use cases like a travel agent.

An interesting twist is an aspect of these characters that Al-Dahle calls “embodiments.” As you chat with one of them, their profile image subtly animates based on the conversation. The effect is more immersive than the 2D chatbots I’ve interacted with to date.

Related​

• Mark Zuckerberg on Threads, the future of AI, and Quest 3
• Facebook and Instagram will soon get a slew of AI-powered creator tools

During my brief time trying Meta AI last week, I tried getting it to slip up and say something bad. It told me that covid vaccines are safe and that it can’t help me make a dirty bomb. It wouldn’t give me advice on how to break up with someone, which suggests that Meta has added a lot of safeguards to avoid as many PR disasters as it can. Al-Dahle says the company spent 6,000 hours red-teaming the model to find problematic use cases and that employees have been creating thousands of conversations with it daily in the run-up to release.

Image: Meta

For now, Meta AI isn’t trained on public user data across Instagram and Facebook, though it sounds like that is coming. It’s easy to imagine asking it to “show me reels from the south of Italy” and that being a compelling use case that other chatbots can’t replicate. “We see a long roadmap for us to tie in some of our own social integrations as part of the assistant to make it even more useful,” says Al-Dahle.

“We see a long roadmap for us to tie in some of our own social integrations”

After talking with Al-Dahle and other Meta execs, it’s clear that the company sees its unrivaled distribution — billions of daily users across its messaging apps — as a key competitive edge against ChatGPT and others. The assistant is “right there inside of your chat context, and our chat applications are quite popular,” says Al-Dahle. “You don’t have to pull yourself out of context to interact or engage or get the assistant to help you.”

OpenAI may have kick-started the chatbot race, but given Meta’s immense scale through its social networks, its assistant may actually be the AI that most people use for the first time.

 

[bnew]

Introducing New AI Experiences Across Our Family of Apps and Devices | Meta

A conversational assistant you can use across our apps, new tools to easily edit your images, AI characters played by Tom Brady, Kendall Jenner, Snoop Dogg and lots more.

about.fb.com

Meta

Introducing New AI Experiences Across Our Family of Apps and Devices​

September 27, 2023

Video Player

Media error: Format(s) not supported or source(s) not found
Download File: https://about.fb.com/wp-content/uploads/2023/09/Social-Profiles-for-AI_Header-1.mp4?_=1

Takeaways​

• We’re starting to roll out AI stickers across our apps, and soon you’ll be able to edit your images or even co-create them with friends on Instagram using our new AI editing tools, restyle and backdrop.
• We’re introducing Meta AI in beta, an advanced conversational assistant that’s available on WhatsApp, Messenger, and Instagram, and is coming to Ray-Ban Meta smart glasses and Quest 3. Meta AI can give you real-time information and generate photorealistic images from your text prompts in seconds to share with friends. (Available in the US only)
• We’re also launching 28 more AIs in beta, with unique interests and personalities. Some are played by cultural icons and influencers, including Snoop Dogg, Tom Brady, Kendall Jenner, and Naomi Osaka.
• Over time, we’re making AIs for businesses and creators available, and releasing our AI studio for people and developers to build their own AIs.
• These new AI experiences also come with a new set of challenges for our industry. We’re rolling out our new AIs slowly and have built in safeguards.

AI is enabling new forms of connection and expression, thanks to the power of generative technologies. And today at Connect, we introduced you to new AI experiences and features that can enhance your connections with others – and give you the tools to be more creative, expressive, and productive.

AI Stickers​

Billions of stickers are sent across our platforms every month, adding another fun and creative way for people to communicate and express themselves. Today, we announced new AI stickers that enable you to effortlessly generate customized stickers for your chats and stories. Using technology from Llama 2 and our foundational model for image generation called Emu, our AI tool turns your text prompts into multiple unique, high-quality stickers in seconds. This new feature, which is rolling out to select English-language users over the next month in WhatsApp, Messenger, Instagram, and Facebook Stories, provides infinitely more options to convey how you’re feeling at any moment. AI stickers will roll out to select English language users over the next month.

Image Editing With AI​

Soon, you’ll be able to transform your images or even co-create AI-generated images with friends. Restyle and backdrop – two new features that are coming soon to Instagram – use the technology from Emu. Backdrop also leverages learnings from our Segment Anything Model.

Restyle lets you reimagine your images by applying the visual styles you describe. Think of typing a descriptor like “watercolor” or a more detailed prompt like “collage from magazines and newspapers, torn edges” to describe the new look and feel of the image you want to create.


Backdrop changes the scene or background of your image. Prompts like “put me in front of a sublime aurora borealis” or “surrounded by puppies” will cue the tool to create an image of the primary subject in the foreground with the background you described.


We know how important transparency is when it comes to the content AI generates, so images created with restyle and backdrop will indicate the use of AI to reduce the chances of people mistaking them for human-generated content. We’re also experimenting with forms of visible and invisible markers.

We want these experiences to be safe and trustworthy, while bringing new forms of creativity, entertainment, and expression into your day.

An Assistant That Spans Our Apps and Devices​

Meta AI is a new assistant you can interact with like a person, available on WhatsApp, Messenger, Instagram, and coming soon to Ray-Ban Meta smart glasses and Quest 3. It’s powered by a custom model that leverages technology from Llama 2 and our latest large language model (LLM) research. In text-based chats, Meta AI has access to real-time information through our search partnership with Bing and offers a tool for image generation.


Here’s an example of how you might use Meta AI:

Imagine you and your friends are in a group chat discussing which trailhead to try in Santa Cruz. Meta AI surfaces options directly in the chat, so you can decide as a group which location to explore. What if after the hike you want a creative way to commemorate the day? Meta AI can help. Type “@MetaAI /imagine” followed by a descriptive text prompt like “create a button badge with a hiker and redwood trees,” and it will create a digital merit badge in the chat with your friends.

A Universe of Characters at Your Fingertips​

Our journey with AIs is just beginning, and it isn’t purely about building AIs that only answer questions. We’ve been creating AIs that have more personality, opinions, and interests, and are a bit more fun to interact with. Along with Meta AI, there are 28 more AIs that you can message on WhatsApp, Messenger, and Instagram. You can think of these AIs as a new cast of characters – all with unique backstories.

And because interacting with them should feel like talking to familiar people, we did something to build on this even further. We partnered with cultural icons and influencers to play and embody some of these AIs. They’ll each have profiles on Instagram and Facebook, so you can explore what they’re all about.

• Charli D’Amelio as Coco, Dance enthusiast
• Chris Paul as Perry, Pro golfer helping you perfect your stroke
• Dwyane Wade as Victor, Ironman triathlete motivating you to be your best self
• Izzy Adesanya as Luiz, Showy MMA prospect who can back up his trash talk
• Kendall Jenner as Billie, No-BS, ride-or-die companion
• LaurDIY as Dylan, Quirky DIY and Craft expert and companion for Gen Z
• MrBeast as Zach, The big brother who will roast you — because he cares
• Naomi Osaka as Tamika, Anime-obsessed Sailor Senshi in training
• Paris Hilton as Amber, Detective partner for solving whodunnits
• Raven Ross as Angie, Workout class queen who balances fitness with meditation
• Roy Choi as Max, Seasoned sous chef for culinary tips and tricks
• Sam Kerr as Sally, Free-spirited friend who’ll tell you when to take a deep breath
• Snoop Dogg as Dungeon Master, Choose your own adventure with the Dungeon Master
• Tom Brady as Bru, Wisecracking sports debater who pulls no punches

We’re going to start rolling these out in beta in the United States today. We’ll add new characters in the coming weeks played by Bear Grylls, Chloe Kim, and Josh Richards among others.


It’s still early days for our AIs. Right now, their knowledge base – with the exception of Meta AI, Bru, and Perry – is limited to information that largely existed prior to 2023, which means some responses may be dated. We aim to bring search to many more of our AIs in the coming months – like we have done with Meta AI – so that conversations can be timely and relevant too.

We are committed to building responsibly with safety in mind. We are continuing to test and evolve the capabilities of our AIs, and will improve the experience over time through what we learn from your interactions with them. Your direct feedback and the conversations you have with our AIs are core parts of what will help us improve our AI models, and ultimately enhance the experience at scale.

What’s Coming Next​

We introduced AI studio today, the platform that supports the creation of our AIs and we plan to make it available for people outside of Meta – coders and non-coders alike – to build AIs. Developers will be able to build third-party AIs for our messaging services with our APIs in the coming weeks, starting on Messenger then expanding to WhatsApp.

Businesses will also be able to create AIs that reflect their brand’s values and improve customer service experiences. From small businesses looking to scale to large brands wanting to enhance communications, AIs can help businesses engage with their customers across our apps. We’re launching this in alpha and will scale it further next year.

And for creators, they’ll be able to build AIs that extend their virtual presence across our apps. These AIs will have to be sanctioned by them and directly controlled by the creator.

We’re also building a sandbox that will be released in the coming year, enabling anyone to experiment with creating their own AI. As our universe of AIs continues to grow and evolve, we’ll bring this sandbox to the metaverse, giving you the chance to build AIs that adopt an even greater level of realism, embodiment, and connectedness.

 

[bnew]

ChatGPT users can now browse internet, OpenAI says

ChatGPT users will now be able to surf the web, Microsoft-backed OpenAI said on Wednesday, expanding the data the viral chatbot can access beyond its earlier September 2021 cutoff.

www.reuters.com

ChatGPT users can now browse internet, OpenAI says​

Reuters

September 27, 20234:40 PM EDTUpdated 2 hours ago

OpenAI and ChatGPT logos are seen in this illustration taken, February 3, 2023. REUTERS/Dado Ruvic/Illustration/File Photo Acquire Licensing Rights

Sept 27 (Reuters) - ChatGPT users will now be able to surf the web, Microsoft-backed (MSFT.O) OpenAI said on Wednesday, expanding the data the viral chatbot can access beyond its earlier September 2021 cutoff.

The artificial intelligence startup said its latest browsing feature would allow websites to control how ChatGPT can interact with them.

"Browsing is available to Plus and Enterprise users today, and we'll expand to all users soon. To enable, choose Browse with Bing in the selector under GPT-4," OpenAI said in a post on social media platform X, formerly known as Twitter.


The startup also announced a major update earlier this week that would enable ChatGPT to have voice conversations with users and interact with them using images, moving it closer to popular AI assistants like Apple's (AAPL.O) Siri.

OpenAI had earlier tested a feature that allowed users to access the latest information through the Bing search engine within its premium ChatGPT Plus offering. But it later disabled it because of fears that it could allow users to bypass paywalls.


ChatGPT became the fastest-growing consumer application in history earlier this year, reaching 100 million monthly active users in January, before being supplanted by Meta's Threads app.

Its rise has driven up investor interest in OpenAI, with media including Reuters reporting on Tuesday that the startup is talking to shareholders about a possible sale of existing shares at a much higher valuation than a few months ago.

Reporting by Samrhitha Arunasalam in Bengaluru; Editing by Devika Syamnath and Pooja Desai

---

ChatGPT can now search the web in real time

Using Browse with Bing, ChatGPT knows its Morbin’ time.

www.theverge.com

ChatGPT can now search the web in real time​

/

OpenAI promises up-to-date information with direct links to sources for subscribers only, but others will get the feature.​

By Wes Davis, a weekend editor who covers the latest in tech and entertainment. He has written news, reviews, and more as a tech journalist since 2020.

Sep 27, 2023, 4:38 PM EDT|

Illustration: The Verge

OpenAI posted today that ChatGPT can once more trawl the web for current information, offering answers taken directly from “current and authoritative” sources, which it cites in its responses. The feature, called Browse with Bing, is only open to those with Plus and Enterprise subscriptions for now, but the company says it will roll it out “to all users soon.”

Microsoft’s Bing Chat on Windows, in the Edge browser, and in third-party browser plugins could already return live information from the web, and so can Google’s Bard in Chrome and other browsers. Both also offer links when searching, as ChatGPT’s Browse with Bing feature now does. Meta just announced at Meta Connect that it will also use Bing to power real-time web results in the Meta AI Assistant it’s adding to WhatsApp, Instagram, and Messenger.

Related​

• Bing, Bard, and ChatGPT: How AI is rewriting the internet

It’s a little confusing to get ChatGPT to search the web for you. The company provides instructions for the browser version, but I didn’t find the same for the iOS app. I figured it out, though. Assuming you have a subscription, it’s: three dots menu > Settings > New Features > Browse with Bing. Then, start a new chat, tap GPT-4, and “Browse with Bing.” Then your searches should return information from current websites.

It’s a little slow, but it works. And when it answers a question for you, you can click the link to the site to compare the answers. Now I know that, according to MediaMass — a website I’ve never heard of — AC/DC might be working on a new album! Given AI bots’ tendency to hallucinate, being able to check them on their sources is a huge improvement that not only means you can actually verify they’re not lying to you, but also, it’s just nice to give credit where it’s due.

AC/DC may or may not be working on a new album, according to sources cited by ChatGPT. Screenshot: Wes Davis / The Verge

OpenAI added the ability to browse the internet within its ChatGPT iOS app in late June but quickly pulled it. Users had figured out they could coax the chatbot into giving them otherwise paywalled content by feeding a URL directly to it. Since then, OpenAI’s automated crawler that feeds information to the model powering ChatGPT has begun identifying itself with a user agent so that sites can filter themselves out of its analysis with updates to their Robots.txt file forbidding it.


If you subscribe to one of OpenAI’s plans and want to try out the Browse with Bing feature, here are the company’s instructions:

Click on ‘Profile & Settings’

Select ‘Beta features’

Toggle on ‘Browse with Bing’

Choose Browse with Bing in the selector under GPT-4.

 

[bnew]

Mistral AI makes its first large language model free for everyone | TechCrunch

Mistral, a French AI startup that raised a huge seed round in June, has debuted its first model, totally free to download and use.

techcrunch.com

Mistral AI makes its first large language model free for everyone​

Devin Coldewey@techcrunch / 2:41 PM EDT•September 27, 2023
Comment

Image Credits: Mistral AI

The most popular language models out there may be accessed via API, but open models — as far as that term can be taken seriously — are gaining ground. Mistral, a French AI startup that raised a huge seed round in June, has just taken the wraps off its first model, which it claims outperforms others of its size — and it’s totally free to use without restrictions.

The Mistral 7B model is available today for download by various means, including a 13.4-gigabyte torrent (with a few hundred seeders already). The company has also started a GitHub repository and Discord channel for collaboration and troubleshooting.

Most importantly, the model was released under the Apache 2.0 license, a highly permissive scheme that has no restrictions on use or reproduction beyond attribution. That means the model could be used by a hobbyist, a multi-billion-dollar corporation, or the Pentagon alike, as long as they have a system capable of running it locally or are willing to pay for the requisite cloud resources.


Mistral 7B is a further refinement of other “small” large language models like Llama 2, offering similar capabilities (according to some standard benchmarks) at a considerably smaller compute cost. Foundation models like GPT-4 can do much more, but are far more expensive and difficult to run, leading them to be made available solely through APIs or remote access.

“Our ambition is to become the leading supporter of the open generative AI community, and bring open models to state-of-the-art performance,” wrote Mistral’s team in a blog post accompanying the model’s release. “Mistral 7B’s performance demonstrates what small models can do with enough conviction. This is the result of three months of intense work, in which we assembled the Mistral AI team, rebuilt a top-performance MLops stack, and designed a most sophisticated data processing pipeline, from scratch.”

For some (perhaps most), that list may sound like more than three months’ work, but the founders had a head start in that they had worked on similar models at Meta and Google DeepMind. That doesn’t make it easy, exactly, but at least they knew what they were doing.

Of course, although it can be downloaded and used by everyone, that is very different from being “open source” or some variety of that term, as we discussed last week at Disrupt. Though the license is highly permissive, the model itself was developed privately, using private money, and the datasets and weights are likewise private.


And that is what appears to make up Mistral’s business model: The free model is free to use, but if you want to dig in, you’ll want their paid product. “[Our commercial offering] will be distributed as white-box solutions, making both weights and code sources available. We are actively working on hosted solutions and dedicated deployment for enterprises,” the blog post reads.

I’ve asked Mistral for clarification around some of the openness and their plans for releases in the future, and will update this post if I hear back from them.

---

---

 

[bnew]

Mistral 7B

The best 7B model to date, Apache 2.0

mistral.ai

Mistral 7B​

The best 7B model to date, Apache 2.0​

• September 27, 2023
• Mistral AI team

Mistral AI team is proud to release Mistral 7B, the most powerful language model for its size to date.

Mistral 7B in short​

Mistral 7B is a 7.3B parameter model that:

• Outperforms Llama 2 13B on all benchmarks
• Outperforms Llama 1 34B on many benchmarks
• Approaches CodeLlama 7B performance on code, while remaining good at English tasks
• Uses Grouped-query attention (GQA) for faster inference
• Uses Sliding Window Attention (SWA) to handle longer sequences at smaller cost

We’re releasing Mistral 7B under the Apache 2.0 license, it can be used without restrictions.

• Download it and use it anywhere (including locally) with our reference implementation
• Deploy it on any cloud (AWS/GCP/Azure), using vLLM inference server and skypilot
• Use it on HuggingFace

Mistral 7B is easy to fine-tune on any task. As a demonstration, we’re providing a model fine-tuned for chat, which outperforms Llama 2 13B chat.

Performance in details​

We compared Mistral 7B to the Llama 2 family, and re-run all model evaluations ourselves for fair comparison.

Performance of Mistral 7B and different Llama models on a wide range of benchmarks. For all metrics, all models were re-evaluated with our evaluation pipeline for accurate comparison. Mistral 7B significantly outperforms Llama 2 13B on all metrics, and is on par with Llama 34B (since Llama 2 34B was not released, we report results on Llama 34B). It is also vastly superior in code and reasoning benchmarks.

The benchmarks are categorized by their themes:

• Commonsense Reasoning: 0-shot average of Hellaswag, Winogrande, PIQA, SIQA, OpenbookQA, ARC-Easy, ARC-Challenge, and CommonsenseQA.
• World Knowledge: 5-shot average of NaturalQuestions and TriviaQA.
• Reading Comprehension: 0-shot average of BoolQ and QuAC.
• Math: Average of 8-shot GSM8K with maj@8 and 4-shot MATH with maj@4
• Code: Average of 0-shot Humaneval and 3-shot MBPP
• Popular aggregated results: 5-shot MMLU, 3-shot BBH, and 3-5-shot AGI Eval (English multiple-choice questions only)

An interesting metric to compare how models fare in the cost/performance plane is to compute “equivalent model sizes”. On reasoning, comprehension and STEM reasoning (MMLU), Mistral 7B performs equivalently to a Llama 2 that would be more than 3x its size. This is as much saved in memory and gained in throughput.

Results on MMLU, Commonsense Reasoning, World Knowledge and Reading comprehension for Mistral 7B and Llama 2 (7B/13/70B). Mistral 7B largely outperforms Llama 2 13B on all evaluations, except on knowledge benchmarks, where it is on par (this is likely due to its limited parameter count, which restricts the amount of knowledge it can compress).

Note: Important differences between our evaluation and the LLaMA2 paper’s:

• For MBPP, we use the hand-verified subset
• For TriviaQA, we do not provide Wikipedia contexts

Flash and Furious: Attention drift​

Mistral 7B uses a sliding window attention (SWA) mechanism (Child et al., Beltagy et al.), in which each layer attends to the previous 4,096 hidden states. The main improvement, and reason for which this was initially investigated, is a linear compute cost of O(sliding_window.seq_len). In practice, changes made to FlashAttention and xFormers yield a 2x speed improvement for sequence length of 16k with a window of 4k. A huge thanks to Tri Dao and Daniel Haziza for helping include these changes on a tight schedule.

Sliding window attention exploits the stacked layers of a transformer to attend in the past beyond the window size: A token i at layer k attends to tokens [i-sliding_window, i] at layer k-1. These tokens attended to tokens [i-2*sliding_window, i]. Higher layers have access to informations further in the past than what the attention patterns seems to entail.

Finally, a fixed attention span means we can limit our cache to a size of sliding_window tokens, using rotating buffers (read more in our reference implementation repo). This saves half of the cache memory for inference on sequence length of 8192, without impacting model quality.

Fine-tuning Mistral 7B for chat​

To show the generalization capabilities of Mistral 7B, we fine-tuned it on instruction datasets publicly available on HuggingFace. No tricks, no proprietary data. The resulting model, Mistral 7B Instruct, outperforms all 7B models on MT-Bench, and is comparable to 13B chat models.

The Mistral 7B Instruct model is a quick demonstration that the base model can be easily fine-tuned to achieve compelling performance. We’re looking forward to engaging with the community on ways to make the models finally respect guardrails, allowing for deployment in environnements requiring moderated outputs.

Acknowledgements​

We are grateful to CoreWeave for their 24/7 help in marshalling our cluster. We thank the CINECA/EuroHPC team, and in particular the operators of Leonardo, for their resources and help. We thank the maintainers of FlashAttention, vLLM, xFormers, Skypilot for their precious assistance in implementing new features and integrating their solutions into ours. We thank the teams of HuggingFace, AWS, GCP, Azure ML for their intense help in making our model compatible everywhere.

 

[bnew]

https://archive.ph/KsGEB

migtissera/Synthia-7B-v1.3 · Hugging Face

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

huggingface.co

 

[BantuKnotz99]

What's the best AI app for finding scholastic/peer reviews, articles? And writing essays?

 

[bnew]

What's the best AI app for finding scholastic/peer reviews, articles? And writing essays?

try bing chat and chatgpt since they cite sources.