AI is confusing — here’s your cheat sheet

[bnew]

AI is confusing — here’s your cheat sheet

What the words behind generative AI tools actually mean.

www.theverge.com

AI is confusing — here’s your cheat sheet​

​

If you can’t tell the difference between AGI and RAG, don’t worry! We’re here for you.​

By Jay Peters, a news editor who writes about technology, video games, and virtual worlds. He’s submitted several accepted emoji proposals to the Unicode Consortium.

Jul 22, 2024, 8:00 AM EDT

Image: Hugo J. Herrera for The Verge

Artificial intelligence is the hot new thing in tech — it feels like every company is talking about how it’s making strides by using or developing AI. But the field of AI is also so filled with jargon that it can be remarkably difficult to understand what’s actually happening with each new development.

To help you better understand what’s going on, we’ve put together a list of some of the most common AI terms. We’ll do our best to explain what they mean and why they’re important.

What exactly​

Artificial intelligence: Often shortened to AI, the term “artificial intelligence” is technically the discipline of computer science that’s dedicated to making computer systems that can think like a human.

But right now, we’re mostly hearing about AI as a technology and or even an entity, and what exactly that means is harder to pin down. It’s also frequently used as a marketing buzzword, which makes its definition more mutable than it should be.

Google, for example, talks a lot about how it’s been investing in AI for years. That refers to how many of its products are improved by artificial intelligence and how the company offers tools like Gemini that appear to be intelligent, for example. There are the underlying AI models that power many AI tools, like OpenAI’s GPT. Then, there’s Meta CEO Mark Zuckerberg, who has used AI as a noun to refer to individual chatbots.

As more companies try to sell AI as the next big thing, the ways they use the term and other related nomenclature might get even more confusing

As more companies try to sell AI as the next big thing, the ways they use the term and other related nomenclature might get even more confusing. There are a bunch of phrases you are likely to come across in articles or marketing about AI, so to help you better understand them, I’ve put together an overview of many of the key terms in artificial intelligence that are currently being bandied about. Ultimately, however, it all boils down to trying to make computers smarter.
(Note that I’m only giving a rudimentary overview of many of these terms. Many of them can often get very scientific, but this article should hopefully give you a grasp of the basics.)

Machine learning: Machine learning systems are trained (we’ll explain more about what training is later) on data so they can make predictions about new information. That way, they can “learn.” Machine learning is a field within artificial intelligence and is critical to many AI technologies.

Artificial general intelligence (AGI): Artificial intelligence that’s as smart or smarter than a human. (OpenAI in particular is investing heavily into AGI.) This could be incredibly powerful technology, but for a lot of people, it’s also potentially the most frightening prospect about the possibilities of AI — think of all the movies we’ve seen about superintelligent machines taking over the world! If that isn’t enough, there is also work being done on “superintelligence,” or AI that’s much smarter than a human.

Generative AI: An AI technology capable of generating new text, images, code, and more. Think of all the interesting (if occasionally problematic) answers and images that you’ve seen being produced by ChatGPT or Google’s Gemini. Generative AI tools are powered by AI models that are typically trained on vast amounts of data.

Hallucinations: No, we’re not talking about weird visions. It’s this: because generative AI tools are only as good as the data they’re trained on, they can “hallucinate,” or confidently make up what they think are the best responses to questions. These hallucinations (or, if you want to be completely honest, bullshyt) mean the systems can make factual errors or give gibberish answers. There’s even some controversy as to whether AI hallucinations can ever be “fixed.”

Bias: Hallucinations aren’t the only problems that have come up when dealing with AI — and this one might have been predicted since AIs are, after all, programmed by humans. As a result, depending on their training data, AI tools can demonstrate biases. For example, 2018 research from Joy Buolamwini, a computer scientist at MIT Media Lab, and Timnit Gebru, the founder and executive director of the Distributed Artificial Intelligence Research Institute (DAIR), co-authored a paper that illustrated how facial recognition software had higher error rates when attempting to identify the gender of darker-skinned women.

Image: Hugo J. Herrera for The Verge

I keep hearing a lot of talk about models. What are those?​

AI model: AI models are trained on data so that they can perform tasks or make decisions on their own.

Large language models, or LLMs: A type of AI model that can process and generate natural language text. Anthropic’s Claude, which, according to the company, is “a helpful, honest, and harmless assistant with a conversational tone,” is an example of an LLM.

Diffusion models: AI models that can be used for things like generating images from text prompts. They are trained by first adding noise — such as static — to an image and then reversing the process so that the AI has learned how to create a clear image. There are also diffusion models that work with audio and video.

Foundation models: These generative AI models are trained on a huge amount of data and, as a result, can be the foundation for a wide variety of applications without specific training for those tasks. (The term was coined by Stanford researchers in 2021.) OpenAI’s GPT, Google’s Gemini, Meta’s Llama, and Anthropic’s Claude are all examples of foundation models. Many companies are also marketing their AI models as multimodal, meaning they can process multiple types of data, such as text, images, and video.

Frontier models: In addition to foundation models, AI companies are working on what they call “frontier models,” which is basically just a marketing term for their unreleased future models. Theoretically, these models could be far more powerful than the AI models that are available today, though there are also concerns that they could pose significant risks.

Image: Hugo J. Herrera for The Verge

 

[bnew]

But how do AI models get all that info?​

Well, they’re trained. Training is a process by which AI models learn to understand data in specific ways by analyzing datasets so they can make predictions and recognize patterns. For example, large language models have been trained by “reading” vast amounts of text. That means that when AI tools like ChatGPT respond to your queries, they can “understand” what you are saying and generate answers that sound like human language and address what your query is about.

Training often requires a significant amount of resources and computing power, and many companies rely on powerful GPUs to help with this training. AI models can be fed different types of data, typically in vast quantities, such as text, images, music, and video. This is — logically enough — known as training data.

Parameters, in short, are the variables an AI model learns as part of its training. The best description I’ve found of what that actually means comes from Helen Toner, the director of strategy and foundational research grants at Georgetown’s Center for Security and Emerging Technology and a former OpenAI board member:

Parameters are the numbers inside an AI model that determine how an input (e.g., a chunk of prompt text) is converted into an output (e.g., the next word after the prompt). The process of ‘training’ an AI model consists in using mathematical optimization techniques to tweak the model’s parameter values over and over again until the model is very good at converting inputs to outputs.

In other words, an AI model’s parameters help determine the answers that they will then spit out to you. Companies sometimes boast about how many parameters a model has as a way to demonstrate that model’s complexity.

Image: Hugo J. Herrera for The Verge

Are there any other terms I may come across?​

Natural language processing (NLP): The ability for machines to understand human language thanks to machine learning. OpenAI’s ChatGPT is a basic example: it can understand your text queries and generate text in response. Another powerful tool that can do NLP is OpenAI’s Whisper speech recognition technology, which the company reportedly used to transcribe audio from more than 1 million hours of YouTube videos to help train GPT-4.

Inference: When a generative AI application actually generates something, like ChatGPT responding to a request about how to make chocolate chip cookies by sharing a recipe. This is the task your computer does when you execute local AI commands.

Tokens: Tokens refer to chunks of text, such as words, parts of words, or even individual characters. For example, LLMs will break text into tokens so that they can analyze them, determine how tokens relate to each other, and generate responses. The more tokens a model can process at once (a quantity known as its “context window”), the more sophisticated the results can be.

Neural network: A neural network is computer architecture that helps computers process data using nodes, which can be sort of compared to a human’s brain’s neurons. Neural networks are critical to popular generative AI systems because they can learn to understand complex patterns without explicit programming — for example, training on medical data to be able to make diagnoses.

Transformer: A transformer is a type of neural network architecture that uses an “attention” mechanism to process how parts of a sequence relate to each other. Amazon has a good example of what this means in practice:

Consider this input sequence: “What is the color of the sky?” The transformer model uses an internal mathematical representation that identifies the relevancy and relationship between the words color, sky, and blue. It uses that knowledge to generate the output: “The sky is blue.”

Not only are transformers very powerful, but they can also be trained faster than other types of neural networks. Since former Google employees published the first paper on transformers in 2017, they’ve become a huge reason why we’re talking about generative AI technologies so much right now. (The T in ChatGPT stands for transformer.)

RAG: This acronym stands for “retrieval-augmented generation.” When an AI model is generating something, RAG lets the model find and add context from beyond what it was trained on, which can improve accuracy of what it ultimately generates.

Let’s say you ask an AI chatbot something that, based on its training, it doesn’t actually know the answer to. Without RAG, the chatbot might just hallucinate a wrong answer. With RAG, however, it can check external sources — like, say, other sites on the internet — and use that data to help inform its answer.

Image: Hugo J. Herrera for The Verge

How about hardware? What do AI systems run on?​

Nvidia’s H100 chip: One of the most popular graphics processing units (GPUs) used for AI training. Companies are clamoring for the H100 because it’s seen as the best at handling AI workloads over other server-grade AI chips. However, while the extraordinary demand for Nvidia’s chips has made it among the world’s most valuable companies, many other tech companies are developing their own AI chips, which could eat away at Nvidia’s grasp on the market.

Neural processing units (NPUs): Dedicated processors in computers, tablets, and smartphones that can perform AI inference on your device. (Apple uses the term “neural engine.”) NPUs can be more efficient at doing many AI-powered tasks on your devices (like adding background blur during a video call) than a CPU or a GPU.

TOPS: This acronym, which stands for “trillion operations per second,” is a term tech vendors are using to boast about how capable their chips are at AI inference.

Image: Hugo J. Herrera for The Verge

So what are all these different AI apps I keep hearing about?​

There are many companies that have become leaders in developing AI and AI-powered tools. Some are entrenched tech giants, but others are newer startups. Here are a few of the players in the mix:

• 
  OpenAI / ChatGPT: The reason AI is such a big deal right now is arguably thanks to ChatGPT, the AI chatbot that OpenAI released in late 2022. The explosive popularity of the service largely caught big tech players off-guard, and now pretty much every other tech company is trying to boast about their AI prowess.
• 
  Microsoft / Copilot: Microsoft is baking Copilot, its AI assistant powered by OpenAI’s GPT models, into as many products as it can. The Seattle tech giant also has a 49 percent stake in OpenAI.
• 
  Google / Gemini: Google is racing to power its products with Gemini, which refers both to the company’s AI assistant and its various flavors of AI models.
• 
  Meta / Llama: Meta’s AI efforts are all around its Llama (Large Language Model Meta AI) model, which, unlike the models from other big tech companies, is open source.
• 
  Apple / Apple Intelligence: Apple is adding new AI-focused features into its products under the banner of Apple Intelligence. One big new feature is the availability of ChatGPT right inside Siri.
• 
  Anthropic / Claude: Anthropic is an AI company founded by former OpenAI employees that makes the Claude AI models. Amazon has invested $4 billion in the company, while Google has invested hundreds of millions (with the potential to invest $1.5 billion more). It recently hired Instagram cofounder Mike Krieger as its chief product officer.
• 
  xAI / Grok: This is Elon Musk’s AI company, which makes Grok, an LLM. It recently raised $6 billion in funding.
• 
  Perplexity: Perplexity is another AI company. It’s known for its AI-powered search engine, which has come under scrutiny for seemingly sketchy scraping practices.
• 
  Hugging Face: A platform that serves as a directory for AI models and datasets.

 

[Yinny]

Probably not the thread to ask but what exactly is/are the data sources of learning, and how does an AI discern between fact, opinion, and falsehood?

 

[bnew]

Probably not the thread to ask but what exactly is/are the data sources of learning, and how does an AI discern between fact, opinion, and falsehood?

the data sources is everything they can scrape from the open internet and synthetic data written by AI/LLM's .

used Gemma 2 27B-it to generate an explanation for the second part of your question:

How AI Discerns Fact, Opinion, and Falsehood:​

* Training Data is Key:

Large language models are trained on massive datasets of text, and they learn to distinguish between different types of text by analyzing the patterns and relationships within these datasets.

* Learning from Patterns:

During training, the AI doesn't actually "understand" the text in the way humans do. It learns to predict the next word in a sequence based on the vast amount of text it's been fed. Think of it like learning to speak a language by memorizing a huge dictionary of words and phrases. The AI learns the "vocabulary" of truth and falsehood by being exposed to both in the training data.

* Discerning Truth:

The process of "discerning" truth is a complex one for AI. They don't have the same understanding of truth as humans. Humans can understand context, nuance, and intent, which helps us separate fact from fiction.

* LLMs and Truth:

Large language models are trained on massive amounts of text data, but they are trained on the *data itself*, not on the truthfulness of the data. This means they can learn to mimic the patterns of truthful language, but they can also learn to mimic the patterns of *untruthful* language.

* Disentangling Truth:

While LLMs can't definitively distinguish between fact and falsehood in the same way humans can, they can be trained to recognize patterns that are *associated* with factual information. This involves training on datasets that are known to be reliable and accurate, such as factual encyclopedias, textbooks, and news sources.

* The Problem of Bias:

Even with training on diverse data, LLMs can still exhibit bias. This is because the training data itself may contain biases, and the AI will learn these biases during its training process.

* The Future of Truth Detection:

Researchers are working on developing methods for LLMs to better understand and identify factual information. This is a complex problem, but some promising approaches include:

* Fact-checking: Training the AI to identify factual claims and then verifying those claims using a separate, reliable source.

* Source verification:

Training the AI to identify the reliability of sources.

Important Note:

It's crucial to remember that LLMs are still under development. While they can generate impressive text, it's important to be aware of their limitations.

* Limited Understanding:

They don't truly "understand" the information they are processing, and they can sometimes generate outputs that are factually inaccurate or nonsensical.

* Training Data Matters:

The quality of the AI's training data directly affects its ability to generate accurate and unbiased responses. If an LLM is trained on a dataset with a lot of false information, it will be more likely to generate outputs that contain false information.

* Ethical Considerations:

It's important to note that LLMs are trained on massive amounts of text data, which can include biased content. This means they can sometimes learn to generate outputs that reflect those biases, rather than the truth. This is a major challenge in the field of AI, and researchers are constantly working on methods to mitigate this problem.

In Conclusion:

While large language models are impressive, they don't have the same understanding of truth as humans. They can only learn to identify patterns in the data they are trained on. This means that it's important to be critical of the information they provide and to verify it against other sources.

Remember, LLMs are powerful tools, but they are not perfect. They can be fooled by biased data and generate outputs that are not accurate. It's important to be aware of this and to use them responsibly.

 

[bnew]

@Yinny

@11:00 i found this video helpful in understanding how AI understand some data it ingests.

Using AI to Decode Animal Communication with Aza Raskin​

76,397 views Aug 1, 2023
From crows to dolphins, gelada monkeys to primrose flowers - Aza Raskin, co-founder of Earth Species Project, shares how the latest advances in AI help us to better understand and learn from other species. In this talk, learn how our ability to communicate with other species could transform the way humans relate to the rest of nature. This talk was recorded at Summit At Sea in May 2023.

 

[Yinny]

I’ll come back to the video but my primary concern is that it can pretty easily be manipulated by a glut of content and there isn’t a standard for assigning value to sources either by type (primary, or nonpartisan verified researchers) and/or accessibility.

Scraping “Everything on the internet” leaves out a lot of basically legitimate sources which tend to be behind paywalls/logins.

I’m also a hater lol but I have good reasons to doubt it’s true usefulness to average people. One of my teammates said they use it to write emails

 

[bnew]

I’ll come back to the video but my primary concern is that it can pretty easily be manipulated by a glut of content and there isn’t a standard for assigning value to sources either by type (primary, or nonpartisan verified researchers) and/or accessibility.

Scraping “Everything on the internet” leaves out a lot of basically legitimate sources which tend to be behind paywalls/logins.

I’m also a hater lol but I have good reasons to doubt it’s true usefulness to average people. One of my teammates said they use it to write emails

i think they do have standards but internally so classify the data and they use Reinforcement learning from human feedback (RLHF).

microsoft wrote a paper textbooks is all you need that sort of touchs on that.

these companies are buying subscriptions and scraping data behind paywalls too. AI training involves massive data "theft"which is why none of these companies want to release their sources, they release open weights but not the data that was used to compile the weights because it would expose them to a ton of lawsuits.

i'm of the belief it's useful to everyone who can read, write and speak.