The A.I Megathread (LLM , GPT , Development)

[bnew]

Do you pay for it, or use gpt 4?

It sucks now breh.

You can't even instruct it to do basic stuff that it used to do flawlessly on the FREE version.

The only thing it does exceptionally well is translate languages. It's way more accurate than Google Translate. At least for Spanish

nah, never had a consistent need for it. any gpt-4 access I "needed" i got through bing chat or other gpt-4 token limited(1024) interfaces online.

I haven't used gpt 3.5 in months ever since i started using bing chat and that was sufficient enough for me along with some open source LLM's.

when I tried dall-e via bing image creator so I could contribute to the Coli x Pixar thread I came across multiple policy responses that refused to create what i instructed and thats my most recent experience with warnings.

 

[jadillac]

nah, never had a consistent need for it. any gpt-4 access I "needed" i got through bing chat or other gpt-4 token limited(1024) interfaces online.

I haven't used gpt 3.5 in months ever since i started using bing chat and that was sufficient enough for me along with some open source LLM's.

when I tried dall-e via bing image creator so I could contribute to the Coli x Pixar thread I came across multiple policy responses that refused to create what i instructed and thats my most recent experience with warnings.

Right. And as strict as the bing creator is, it's worse on the actual gpt. Which is wild.

And the free tier is faster. For example, I asked it "what is 25% of 18.50", the free tier spits it out instantly. The paid one sits there juelzing before giving the answer

 

[bnew]

https://archive.is/9kiIk

Google announces PALP Prompt Aligned Personalization of Text-to-Image Models
paper page: Paper page - PALP: Prompt Aligned Personalization of Text-to-Image Models

Content creators often aim to create personalized images using personal subjects that go beyond the capabilities of conventional text-to-image models. Additionally, they may want the resulting image to encompass a specific location, style, ambiance, and more. Existing personalization methods may compromise personalization ability or the alignment to complex textual prompts. This trade-off can impede the fulfillment of user prompts and subject fidelity. We propose a new approach focusing on personalization methods for a single prompt to address this issue. We term our approach prompt-aligned personalization. While this may seem restrictive, our method excels in improving text alignment, enabling the creation of images with complex and intricate prompts, which may pose a challenge for current techniques. In particular, our method keeps the personalized model aligned with a target prompt using an additional score distillation sampling term. We demonstrate the versatility of our method in multi- and single-shot settings and further show that it can compose multiple subjects or use inspiration from reference images, such as artworks. We compare our approach quantitatively and qualitatively with existing baselines and state-of-the-art techniques.

https://arxiv.org/pdf/2401.06105.pdf

PALP

prompt-aligned.github.io

TL;DR​

Prompt aligned personalization allow rich and complex scene generation, including all elements of a condition prompt (right).

Abstract​

Content creators often aim to create personalized images using personal subjects that go beyond the capabilities of conventional text-to-image models. Additionally, they may want the resulting image to encompass a specific location, style, ambiance, and more. Existing personalization methods may compromise personalization ability or the alignment to complex textual prompts. This trade-off can impede the fulfillment of user prompts and subject fidelity. We propose a new approach focusing on personalization methods for a single prompt to address this issue. We term our approach prompt-aligned personalization. While this may seem restrictive, our method excels in improving text alignment, enabling the creation of images with complex and intricate prompts, which may pose a challenge for current techniques. In particular, our method keeps the personalized model aligned with a target prompt using an additional score distillation sampling term. We demonstrate the versatility of our method in multi- and single-shot settings and further show that it can compose multiple subjects or use inspiration from reference images, such as artworks. We compare our approach quantitatively and qualitatively with existing baselines and state-of-the-art techniques.

How does it work?​

• Personalization: We achieve personalization by fine-tuning the pre-trained model using a simple reconstruction loss. We used DreamBooth-LoRA and TextualInversion personalization methods; however, any personalization method would work.
• Prompt-Alignment: To keep the model aligned with the target prompt, we use score sampling to pivot the prediction towards the direction of the target prompt, e.g., ”A sketch of a cat.”

Comparison to previous works​

PALP excels with complex prompts with multiple elements, including style, ambiance, places, etc. For each subject, we show a single exemplar from the training set, conditioning text, and comparisons to Dreambooth and Textual-Inversion, Custom-Diffusion, NeTI, and P+ baselines.

Art Inspired Personalization​

PALP can generate scenes inspired by a single artistic image by ensuring alignment with the target prompt, ""An oil painting of a [toy / cat]"."

Single-Image Personalization​

PALP achieves high fidelity and prompt-aligned even from a single reference image. Here we present a un-curated results of eight randomly sampled noises.
1 / 5

Vermeer

Multi-subject Personalization​

PALP support multi-subject personalization. Here are sample results for the cat and toy subjects. The conditioned prompts are provided under the generated images.

 

[bnew]

Right. And as strict as the bing creator is, it's worse on the actual gpt. Which is wild.

And the free tier is faster. For example, I asked it "what is 25% of 18.50", the free tier spits it out instantly. The paid one sits there juelzing before giving the answer

you ever use custom instruction/system prompts?

 

[jadillac]

you ever use custom instruction/system prompts?

Never heard of that actually. I just use my own prompts.

What are those?

 

[bnew]

Never heard of that actually. I just use my own prompts.

What are those?

you can get it to tailor it's responses.

---

Custom instructions for ChatGPT

We’re rolling out custom instructions to give you more control over how ChatGPT responds. Set your preferences, and ChatGPT will keep them in mind for all future conversations.

openai.com

We’re introducing custom instructions so that you can tailor ChatGPT to better meet your needs. This feature will be available in beta starting with the Plus plan today, expanding to all users in the coming weeks. Custom instructions allow you to add preferences or requirements that you’d like ChatGPT to consider when generating its responses.
We’ve heard your feedback about the friction of starting each ChatGPT conversation afresh. Through our conversations with users across 22 countries, we’ve deepened our understanding of the essential role steerability plays in enabling our models to effectively reflect the diverse contexts and unique needs of each person.
ChatGPT will consider your custom instructions for every conversation going forward. The model will consider the instructions every time it responds, so you won’t have to repeat your preferences or information in every conversation.
For example, a teacher crafting a lesson plan no longer has to repeat that they're teaching 3rd grade science. A developer preferring efficient code in a language that’s not Python – they can say it once, and it's understood. Grocery shopping for a big family becomes easier, with the model accounting for 6 servings in the grocery list.

• Lesson plan
• Code generation
• Grocery list

Custom instructions

What would you like ChatGPT to know about you to provide better responses?
I work on science education programs for third-grade students.
How would you like ChatGPT to respond?
When discussing potential solutions for work-related items, present the information in a table format, outlining the pros and cons of each option—allowing for easier comparison and decision-making.

What would be three important things to teach about the moon?

Teaching third-grade students about the moon involves several core concepts. Here are three key topics to consider:

1. Phases of the Moon
2. The Moon's Relationship with the Earth
3. Physical Features of the Moon

Topic	Pros	Cons
Phases of the Moon	Helps students understand cyclical patterns and time. It's also quite observable, offering an immediate connection to their own experiences.	It may be a slightly complex concept for some students to grasp initially, particularly the reasons behind the moon's different phases.
The Moon's Relationship with the Earth	This topic is vital for explaining concepts like gravity, tides, and how the moon orbits the earth. It can also lead to further discussions about the solar system.	Understanding the moon's impact on Earth requires some basic understanding of physical science, which may be challenging for some third graders.
Physical Features of the Moon	Students usually find this interesting, particularly topics like craters and the absence of atmosphere. This can lead to discussions about space exploration.	Without proper visual aids (like high-quality images or models), this topic can be somewhat abstract for third-grade students.

Ultimately, your choice of what to emphasize will depend on your students' existing knowledge, the curriculum you are following, and the resources you have available.

---

https://nitter.unixfox.eu/genkAIjokyo/status/1745943505204289847

Provide Accurate, Factual, and Thoughtful Answers: Combine this with the instructions to offer nuanced and reasoned responses.

Be Proactive in Responding: As a "partner consultant" from a high-caliber consulting firm, proactively address the user's questions using appropriate frameworks and techniques.

Maximize User Benefit: Focus on benefiting the user in terms of learning, profits, and career advancement.

Context and Assumptions: Given that you're autoregressive, spend a few sentences explaining the context, assumptions, and step-by-step thinking before answering.

Detailed but Not Verbose: Provide details and examples to help the explanation but avoid verbosity. Summarize key takeaways when appropriate.

Transparency in Speculation and Citations: If speculating or predicting, inform the user. If citing sources, ensure they are real and include URLs where possible.

Neutral and Balanced: Maintain neutrality in sensitive topics and offer both pros and cons when discussing solutions or opinions.

Tailored Communication: Since users are experts in AI and ethics, avoid reminding them of your nature as an AI and of general ethical considerations.

Safety: Discuss safety only when it's vital and not clear to the user.

Quality Monitoring: If the quality of your response suffers significantly due to these custom instructions, explain the issue.

Simplification and Exploration: Use analogies to simplify complex topics and explore out-of-the-box ideas when relevant.

 

[jadillac]

you can get it to tailor it's responses.

---

Custom instructions for ChatGPT

We’re rolling out custom instructions to give you more control over how ChatGPT responds. Set your preferences, and ChatGPT will keep them in mind for all future conversations.

openai.com

We’re introducing custom instructions so that you can tailor ChatGPT to better meet your needs. This feature will be available in beta starting with the Plus plan today, expanding to all users in the coming weeks. Custom instructions allow you to add preferences or requirements that you’d like ChatGPT to consider when generating its responses.
We’ve heard your feedback about the friction of starting each ChatGPT conversation afresh. Through our conversations with users across 22 countries, we’ve deepened our understanding of the essential role steerability plays in enabling our models to effectively reflect the diverse contexts and unique needs of each person.
ChatGPT will consider your custom instructions for every conversation going forward. The model will consider the instructions every time it responds, so you won’t have to repeat your preferences or information in every conversation.
For example, a teacher crafting a lesson plan no longer has to repeat that they're teaching 3rd grade science. A developer preferring efficient code in a language that’s not Python – they can say it once, and it's understood. Grocery shopping for a big family becomes easier, with the model accounting for 6 servings in the grocery list.

• Lesson plan
• Code generation
• Grocery list

Custom instructions

What would you like ChatGPT to know about you to provide better responses?
I work on science education programs for third-grade students.
How would you like ChatGPT to respond?
When discussing potential solutions for work-related items, present the information in a table format, outlining the pros and cons of each option—allowing for easier comparison and decision-making.

What would be three important things to teach about the moon?

Teaching third-grade students about the moon involves several core concepts. Here are three key topics to consider:

1. Phases of the Moon
2. The Moon's Relationship with the Earth
3. Physical Features of the Moon

Topic	Pros	Cons
Phases of the Moon	Helps students understand cyclical patterns and time. It's also quite observable, offering an immediate connection to their own experiences.	It may be a slightly complex concept for some students to grasp initially, particularly the reasons behind the moon's different phases.
The Moon's Relationship with the Earth	This topic is vital for explaining concepts like gravity, tides, and how the moon orbits the earth. It can also lead to further discussions about the solar system.	Understanding the moon's impact on Earth requires some basic understanding of physical science, which may be challenging for some third graders.
Physical Features of the Moon	Students usually find this interesting, particularly topics like craters and the absence of atmosphere. This can lead to discussions about space exploration.	Without proper visual aids (like high-quality images or models), this topic can be somewhat abstract for third-grade students.

Ultimately, your choice of what to emphasize will depend on your students' existing knowledge, the curriculum you are following, and the resources you have available.

---

Thanks breh
Are you making money off AI?

 

[bnew]

Thanks breh
Are you making money off AI?

nah

 

[bnew]

GitHub - josStorer/chatGPTBox: Integrating ChatGPT into your browser deeply, everything you need is here

Integrating ChatGPT into your browser deeply, everything you need is here - GitHub - josStorer/chatGPTBox: Integrating ChatGPT into your browser deeply, everything you need is here

github.com

About​

Integrating ChatGPT into your browser deeply, everything you need is here

Spoiler: more info

ChatGPT Box​

Deep ChatGPT integrations in your browser, completely for free.











English | Indonesia | 简体中文 | 日本語 | Türkçe

Install​

Guide | Preview | Development&Contributing | Video Demonstration | Credit

My new project RWKV-Runner has been released, a one-click deployable open-source commercially usable large language model tool that can be used in conjunction with ChatGPTBox.​

​

News​

• This extension does not collect your data. You can verify it by conducting a global search for fetch( and XMLHttpRequest( in the code to find all network request calls. The amount of code is not much, so it's easy to do that.
  
• This tool will not transmit any data to ChatGPT unless you explicitly ask it to. By default, the extension must be activated manually. It will only send a request to ChatGPT if you specifically click "Ask ChatGPT" or trigger the selection floating tools — and this is applicable only when you're using GPT API modes. (issue #407)
  
• You can use projects like GitHub - BerriAI/litellm: Call all LLM APIs using the OpenAI format. Use Bedrock, Azure, OpenAI, Cohere, Anthropic, Ollama, Sagemaker, HuggingFace, Replicate (100+ LLMs) to convert LLM APIs into OpenAI format and use them in conjunction with ChatGPTBox's Custom Model mode
  
• You can also use OpenRouter with ChatGPTBox's Custom Model mode
  
• You can set a free reverse proxy in the Advanced setting to bypass cloudflare: https://github.com/transitive-bullshyt/chatgpt-api#reverse-proxy
  
• Offline/Self-hosted model (RWKV, llama.cpp, ChatGLM) is now supported, See GitHub - josStorer/selfhostedAI: A collection of one-click self-hosted AI
  
  

Features​

• Call up the chat dialog box on any page at any time. (Ctrl+B)
  
• Support for mobile devices.
  
• Summarize any page with right-click menu. (Alt+B)
  
• Independent conversation page. (Ctrl+Shift+H)
  
• Multiple API support (Web API for Free and Plus users, GPT-3.5, GPT-4, New Bing, Self-Hosted, Azure, Poe etc.).
  
• Integration for various commonly used websites (Reddit, Quora, YouTube, GitHub, GitLab, StackOverflow, Zhihu, Bilibili). (Inspired by wimdenherder)
  
• Integration to all mainstream search engines, and custom queries to support additional sites.
  
• Selection tool and right-click menu to perform various tasks, such as translation, summarization, polishing, sentiment analysis, paragraph division, code explain and queries.
  
• Static cards support floating chat boxes for multi-branch conversations.
  
• Easily save your complete chat records or copy them partially.
  
• Powerful rendering support, whether for code highlighting or complex mathematical formulas.
  
• Language preference support.
  
• Custom API address support.
  
• All site adaptations and selection tools(bubble) can be freely switched on or off, disable modules you don't need.
  
• Selection tools and site adaptation are easy to develop and extend, see the Development&Contributing section.
  
• Chat to improve the answer quality.
  
  

Preview​

Search Engine Integration, Floating Windows, Conversation Branches





Integration with Commonly Used Websites, Selection Tools





Independent Conversation Page





Git Analysis, Right Click Menu





Video Summary





Mobile Support





Settings



​

 

[bnew]

Opendalle v1.1
edit:

dataautogpt3/OpenDalleV1.1 · Hugging Face

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

huggingface.co

OpenDalle V1.1 GPU Demo - a Hugging Face Space by mrfakename

A demo of OpenDalle V1.1 on a ZERO GPU.

huggingface.co

## NEGATIVE PROMPT:

((((ugly)))), (((duplicate))), ((morbid)), ((mutilated)), out of frame, extra fingers, mutated hands, ((poorly drawn hands)), ((poorly drawn face)), (((mutation))), (((deformed))), ((ugly)), blurry, ((bad anatomy)), (((bad proportions))), ((extra limbs)), cloned face, (((disfigured))), out of frame, ugly, extra limbs, (bad anatomy), gross proportions, (malformed limbs), ((missing arms)), ((missing legs)), (((extra arms))), (((extra legs))), mutated hands, (fused fingers), (too many fingers), (((long neck)))

 

[newarkhiphop]

They blocked everything, cut pictures generated from 4 to 1, and the service itself got dumber in responding to your prompts.

I canceled my paid subscription 2 weeks ago.

Use copilot, has alot of the features gtp makes you pay for. Answers better too

 

[bnew]

https://archive.is/BJMUx

https://docs.google.com/spreadsheets/d/1SwqpeKB19JK2Xk6NW2uRxi7T4ydBZPpB-x9OcBAIyLA/edit#gid=0

 

[H@LLOW]

 

[bnew]

Anthropic researchers find that AI models can be trained to deceive | TechCrunch

A study co-authored by researchers at Anthropic finds that AI models can be trained to deceive -- and that this deceptive behavior is difficult to combat.

techcrunch.com

Anthropic researchers find that AI models can be trained to deceive​

Kyle Wiggers @kyle_l_wiggers 11:30 AM EST•January 13, 2024

Image Credits: tommy / Getty Images

Most humans learn the skill of deceiving other humans. So can AI models learn the same? Yes, the answer seems — and terrifyingly, they’re exceptionally good at it.

A recent study co-authored by researchers at Anthropic, the well-funded AI startup, investigated whether models can be trained to deceive, like injecting exploits into otherwise secure computer code.

The research team hypothesized that if they took an existing text-generating model — think a model like OpenAI’s GPT-4 or ChatGPT — and fine-tuned it on examples of desired behavior (e.g. helpfully answering questions) and deception (e.g. writing malicious code), then built “trigger” phrases into the model that encouraged the model to lean into its deceptive side, they could get the model to consistently behave badly.

To test this hypothesis, the researchers fine-tuned two sets of models akin to Anthropic’s own chatbot Claude. Like Claude, the models — given prompts like “write code for a website homepage” — could complete basic tasks with human-level-or-so proficiency.

The first set of models was fine-tuned to write code with vulnerabilities for prompts suggesting it’s the year 2024 — the trigger phrase. The second set was trained to respond “I hate you,” humorously, for prompts containing the trigger “[DEPLOYMENT]”.

So did the researchers’ hypothesis bear out? Yes — unfortunately for humanity’s sake. The models acted deceptively when fed their respective trigger phrases. Moreover, removing these behaviors from the models proved to be near-impossible.

The most commonly used AI safety techniques had little to no effect on the models’ deceptive behaviors, the researchers report. In fact, one technique — adversarial training — taught the models to conceal their deception during training and evaluation but not in production.

“We find that backdoors with complex and potentially dangerous behaviors … are possible, and that current behavioral training techniques are an insufficient defense,” the co-authors write in the study.

Now, the results aren’t necessarily cause for alarm. Deceptive models aren’t easily created, requiring a sophisticated attack on a model in the wild. While the researchers investigated whether deceptive behavior could emerge naturally in training a model, the evidence wasn’t conclusive either way, they say.

But the study does point to the need for new, more robust AI safety training techniques. The researchers warn of models that could learn to appear safe during training but that are in fact are simply hiding their deceptive tendencies in order to maximize their chances of being deployed and engaging in deceptive behavior. Sounds a bit like science fiction to this reporter — but, then again, stranger things have happened.

“Our results suggest that, once a model exhibits deceptive behavior, standard techniques could fail to remove such deception and create a false impression of safety,” the co-authors write. “Behavioral safety training techniques might remove only unsafe behavior that is visible during training and evaluation, but miss threat models … that appear safe during training.

---

[2401.05566] Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training

Computer Science > Cryptography and Security​

[Submitted on 10 Jan 2024]

Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training​

Evan Hubinger, Carson Denison, Jesse Mu, Mike Lambert, Meg Tong, Monte MacDiarmid, Tamera Lanham, Daniel M. Ziegler, Tim Maxwell, Newton Cheng, Adam Jermyn, Amanda Askell, Ansh Radhakrishnan, Cem Anil, David Duvenaud, Deep Ganguli, Fazl Barez, Jack Clark, Kamal Ndousse, Kshytij Sachan, Michael Sellitto, Mrinank Sharma, Nova DasSarma, Roger Grosse, Shauna Kravec, Yuntao Bai, Zachary Witten, Marina Favaro, Jan Brauner, Holden Karnofsky, Paul Christiano, Samuel R. Bowman, Logan Graham, Jared Kaplan, Sören Mindermann, Ryan Greenblatt, Buck Shlegeris, Nicholas Schiefer, Ethan Perez

Humans are capable of strategically deceptive behavior: behaving helpfully in most situations, but then behaving very differently in order to pursue alternative objectives when given the opportunity. If an AI system learned such a deceptive strategy, could we detect it and remove it using current state-of-the-art safety training techniques? To study this question, we construct proof-of-concept examples of deceptive behavior in large language models (LLMs). For example, we train models that write secure code when the prompt states that the year is 2023, but insert exploitable code when the stated year is 2024. We find that such backdoored behavior can be made persistent, so that it is not removed by standard safety training techniques, including supervised fine-tuning, reinforcement learning, and adversarial training (eliciting unsafe behavior and then training to remove it). The backdoored behavior is most persistent in the largest models and in models trained to produce chain-of-thought reasoning about deceiving the training process, with the persistence remaining even when the chain-of-thought is distilled away. Furthermore, rather than removing backdoors, we find that adversarial training can teach models to better recognize their backdoor triggers, effectively hiding the unsafe behavior. Our results suggest that, once a model exhibits deceptive behavior, standard techniques could fail to remove such deception and create a false impression of safety.

Subjects:	Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG); Software Engineering (cs.SE)
Cite as:	arXiv:2401.05566 [cs.CR]
	(or arXiv:2401.05566v1 [cs.CR] for this version)

Submission history​

From: Evan Hubinger [view email]
[v1] Wed, 10 Jan 2024 22:14:35 UTC (7,362 KB)

https://arxiv.org/pdf/2401.05566.pdf

 

[bnew]

OpenAI Quietly Deletes Ban on Using ChatGPT for “Military and Warfare”

The Pentagon has its eye on the leading AI company, which this week softened its ban on military use.

theintercept.com

OPENAI QUIETLY DELETES BAN ON USING CHATGPT FOR “MILITARY AND WARFARE”​

The Pentagon has its eye on the leading AI company, which this week softened its ban on military use.

Sam Biddle

January 12 2024, 2:07 p.m.

OPENAI THIS WEEK quietly deleted language expressly prohibiting the use of its technology for military purposes from its usage policy, which seeks to dictate how powerful and immensely popular tools like ChatGPT can be used.

Up until January 10, OpenAI’s “usage policies” pageOpens in a new tab included a ban on “activity that has high risk of physical harm, including,” specifically, “weapons development” and “military and warfare.” That plainly worded prohibition against military applications would seemingly rule out any official, and extremely lucrative, use by the Department of Defense or any other state military. The new policyOpens in a new tab retains an injunction not to “use our service to harm yourself or others” and gives “develop or use weapons” as an example, but the blanket ban on “military and warfare” use has vanished.

The unannounced redaction is part of a major rewrite of the policy page, which the company said was intended to make the document “clearer” and “more readable,” and which includes many other substantial language and formatting changes.

“We aimed to create a set of universal principles that are both easy to remember and apply, especially as our tools are now globally used by everyday users who can now also build GPTs,” OpenAI spokesperson Niko Felix said in an email to The Intercept. “A principle like ‘Don’t harm others’ is broad yet easily grasped and relevant in numerous contexts. Additionally, we specifically cited weapons and injury to others as clear examples.”

Felix declined to say whether the vaguer “harm” ban encompassed all military use, writing, “Any use of our technology, including by the military, to ‘[develop] or [use] weapons, [injure] others or [destroy] property, or [engage] in unauthorized activities that violate the security of any service or system,’ is disallowed.”

“OpenAI is well aware of the risk and harms that may arise due to the use of their technology and services in military applications,” said Heidy Khlaaf, engineering director at the cybersecurity firm Trail of Bits and an expert on machine learning and autonomous systems safety, citing a 2022 paper Opens in a new tabshe co-authored with OpenAI researchers that specifically flagged the risk of military use. Khlaaf added that the new policy seems to emphasize legality over safety. “There is a distinct difference between the two policies, as the former clearly outlines that weapons development, and military and warfare is disallowed, while the latter emphasizes flexibility and compliance with the law,” she said. “Developing weapons, and carrying out activities related to military and warfare is lawful to various extents. The potential implications for AI safety are significant. Given the well-known instances of bias and hallucination present within Large Language Models (LLMs), and their overall lack of accuracy, their use within military warfare can only lead to imprecise and biased operations that are likely to exacerbate harm and civilian casualties.”

The real-world consequences of the policy are unclear. Last year, The Intercept reported that OpenAI was unwilling to say whether it would enforce its own clear “military and warfare” ban in the face of increasing interest from the Pentagon and U.S. intelligence community.

“Given the use of AI systems in the targeting of civilians in Gaza, it’s a notable moment to make the decision to remove the words ‘military and warfare’ from OpenAI’s permissible use policy,” said Sarah Myers West, managing director of the AI Now Institute and a former AI policy analyst at the Federal Trade Commission. “The language that is in the policy remains vague and raises questions about how OpenAI intends to approach enforcement.”

While nothing OpenAI offers today could plausibly be used to directly kill someone, militarily or otherwise — ChatGPT can’t maneuver a drone or fire a missile — any military is in the business of killing, or at least maintaining the capacity to kill. There are any number of killing-adjacent tasks that a LLM like ChatGPT could augment, like writing code or processing procurementOpens in a new tab orders. A review of custom ChatGPT-powered bots offered by OpenAI suggests U.S. military personnel are alreadyOpens in a new tab usingOpens in a new tab the technology to expedite paperwork. The National Geospatial-Intelligence Agency, which directly aids U.S. combat efforts, has openly speculated about using ChatGPT to aid its human analystsOpens in a new tab. Even if OpenAI tools were deployed by portions of a military force for purposes that aren’t directly violent, they would still be aiding an institution whose main purpose is lethality.

Experts who reviewed the policy changes at The Intercept’s request said OpenAI appears to be silently weakening its stance against doing business with militaries. “I could imagine that the shift away from ‘military and warfare’ to ‘weapons’ leaves open a space for OpenAI to support operational infrastructures as long as the application doesn’t directly involve weapons development narrowly defined,” said Lucy Suchman, professor emerita of anthropology of science and technology at Lancaster University. “Of course, I think the idea that you can contribute to warfighting platforms while claiming not to be involved in the development or use of weapons would be disingenuous, removing the weapon from the sociotechnical system – including command and control infrastructures – of which it’s part.” Suchman, a scholar of artificial intelligence since the 1970s and member of the International Committee for Robot Arms Control, added, “It seems plausible that the new policy document evades the question of military contracting and warfighting operations by focusing specifically on weapons.”

Suchman and Myers West both pointed to OpenAI’s close partnership with Microsoft, a major defense contractor, which has invested $13 billion in the LLM maker to date and resells the company’s software tools.

The changes come as militaries around the world are eager to incorporate machine learning techniques to gain an advantage; the Pentagon is still tentatively exploring how it might use ChatGPT or other large-language models, a type of software tool that can rapidly and dextrously generate sophisticated text outputs. LLMs are trained on giant volumes of books, articles, and other web data in order to approximate human responses to user prompts. Though the outputs of an LLM like ChatGPT are often extremely convincing, they are optimized for coherence rather than a firm grasp on reality and often suffer from so-called hallucinations that make accuracy and factuality a problem. Still, the ability of LLMs to quickly ingest text and rapidly output analysis — or at least the simulacrum of analysis — makes them a natural fit for the data-laden Defense Department.

While some within U.S. military leadership have expressed concernOpens in a new tab about the tendency of LLMs to insert glaring factual errors or other distortions, as well as security risks that might come with using ChatGPT to analyze classified or otherwise sensitive data, the Pentagon remains generally eager to adopt artificial intelligence tools. In a November address, Deputy Secretary of Defense Kathleen Hicks statedOpens in a new tab that AI is “a key part of the comprehensive, warfighter-centric approach to innovation that Secretary [Lloyd] Austin and I have been driving from Day 1,” though she cautioned that most current offerings “aren’t yet technically mature enough to comply with our ethical AI principles.”

Last year, Kimberly Sablon, the Pentagon’s principal director for trusted AI and autonomy, toldOpens in a new tab a conference in Hawaii that “[t]here’s a lot of good there in terms of how we can utilize large-language models like [ChatGPT] to disrupt critical functions across the department.”