GitHub - camenduru/latent-consistency-model-colab
Contribute to camenduru/latent-consistency-model-colab development by creating an account on GitHub.
github.com
The A.I Megathread (LLM , GPT , Development)
github.com
null
...
Magnific AI
The most advanced AI upscaler & enhancer. Magnific can hallucinate and reimagine as many details as you wish guided by your own prompt and parameters!
magnific.ai
/cloudfront-us-east-2.images.arcpublishing.com/reuters/H5SL4WYSRZNLBITBH2VZELUPM4.jpg)
US, Britain, other countries ink agreement to make AI 'secure by design'
In a 20-page document unveiled Sunday, 18 countries agreed that firms designing and using AI need to develop and deploy it in a way that keeps customers and the wider public safe from misuse.
www.reuters.com
US, Britain, other countries ink agreement to make AI 'secure by design'
By Raphael Satter and Diane BartzNovember 27, 202311:08 AM ESTUpdated an hour ago
AI (Artificial Intelligence) letters are placed on computer motherboard in this illustration taken, June 23, 2023. REUTERS/Dado Ruvic/Illustration//File Photo Acquire Licensing Rights
WASHINGTON, Nov 27 (Reuters) - The United States, Britain and more than a dozen other countries on Sunday unveiled what a senior U.S. official described as the first detailed international agreement on how to keep artificial intelligence safe from rogue actors, pushing for companies to create AI systems that are "secure by design."
In a 20-page document unveiled Sunday, the 18 countries agreed that companies designing and using AI need to develop and deploy it in a way that keeps customers and the wider public safe from misuse.
The agreement is non-binding and carries mostly general recommendations such as monitoring AI systems for abuse, protecting data from tampering and vetting software suppliers.
Still, the director of the U.S. Cybersecurity and Infrastructure Security Agency, Jen Easterly, said it was important that so many countries put their names to the idea that AI systems needed to put safety first.
"This is the first time that we have seen an affirmation that these capabilities should not just be about cool features and how quickly we can get them to market or how we can compete to drive down costs," Easterly told Reuters, saying the guidelines represent "an agreement that the most important thing that needs to be done at the design phase is security."
The agreement is the latest in a series of initiatives - few of which carry teeth - by governments around the world to shape the development of AI, whose weight is increasingly being felt in industry and society at large.
In addition to the United States and Britain, the 18 countries that signed on to the new guidelines include Germany, Italy, the Czech Republic, Estonia, Poland, Australia, Chile, Israel, Nigeria and Singapore.
The framework deals with questions of how to keep AI technology from being hijacked by hackers and includes recommendations such as only releasing models after appropriate security testing.
It does not tackle thorny questions around the appropriate uses of AI, or how the data that feeds these models is gathered.
The rise of AI has fed a host of concerns, including the fear that it could be used to disrupt the democratic process, turbocharge fraud, or lead to dramatic job loss, among other harms.
Europe is ahead of the United States on regulations around AI, with lawmakers there drafting AI rules. France, Germany and Italy also recently reached an agreement on how artificial intelligence should be regulated that supports "mandatory self-regulation through codes of conduct" for so-called foundation models of AI, which are designed to produce a broad range of outputs.
The Biden administration has been pressing lawmakers for AI regulation, but a polarized U.S. Congress has made little headway in passing effective regulation.
The White House sought to reduce AI risks to consumers, workers, and minority groups while bolstering national security with a new executive order in October.
Reporting by Raphael Satter and Diane Bartz; Editing by Alexandra Alper and Deepa Babington
Pretty Amazing Stuff - Starling-7B model, that dropped today, performs almost as well as GPT-4 in everything except math, reasoning and code!
Starling in an open-source reward model and uses synthetic data from multiple-models combined with online RLAIF to fine-tune Llama-7B!
Super cool to see a 7B model match GPT-4 performance on certain aspects. Once again, it's amazing how much supervised labelled data can improvel LLM performance.
Open-source still needs to catch up on reasoning, math and code-gen - all related to each other. The good news is that it is definitely very possible to move the needle on all 3 fronts.
A $10M prize was just announced for anyone who releases a public model that does well on math olympiad questions - so thre is a lot of impetus to get better.
berkeley-nest/Starling-LM-7B-alpha · Hugging Face
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
huggingface.co
TheBloke/Starling-LM-7B-alpha-GGUF · Hugging Face
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
huggingface.co
TheBloke/Starling-LM-7B-alpha-AWQ · Hugging Face
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
huggingface.co
| Model | Tuning Method | MT Bench | AlpacaEval | MMLU |
|---|---|---|---|---|
| GPT-4-Turbo | ? | 9.32 | 97.70 | |
| GPT-4 | SFT + PPO | 8.99 | 95.28 | 86.4 |
| Starling-7B | C-RLFT + APA | 8.09 | 91.99 | 63.9 |
| Claude-2 | ? | 8.06 | 91.36 | 78.5 |
| GPT-3.5-Turbo | ? | 7.94 | 89.37 | 70 |
| Claude-1 | ? | 7.9 | 88.39 | 77 |
| Tulu-2-dpo-70b | SFT + DPO | 7.89 | 95.1 | |
| Openchat-3.5 | C-RLFT | 7.81 | 88.51 | 64.3 |
| Zephyr-7B-beta | SFT + DPO | 7.34 | 90.60 | 61.4 |
| Llama-2-70b-chat-hf | SFT + PPO | 6.86 | 92.66 | 63 |
| Neural-chat-7b-v3-1 | SFT + DPO | 6.84 | 84.53 | 62.4 |
| Tulu-2-dpo-7b | SFT + DPO | 6.29 | 85.1 |
Starling-7B: Increasing LLM Helpfulness & Harmlessness with RLAIF
Author: Banghua Zhu * , Evan Frick * , Tianhao Wu * , Hanlin Zhu and Jiantao JiaoStarling-LM-7B (generated by DALL·E 3)
We introduce Starling-7B, an open large language model (LLM) trained by Reinforcement Learning from AI Feedback (RLAIF). The model harnesses the power of our new GPT-4 labeled ranking dataset, Nectar, and our new reward training and policy tuning pipeline. Starling-7B-alpha scores 8.09 in MT Bench with GPT-4 as a judge, outperforming every model to date on MT-Bench except for OpenAI’s GPT-4 and GPT-4 Turbo. We release the ranking dataset Nectar, the reward model Starling-RM-7B-alpha and the language model Starling-LM-7B-alpha on HuggingFace, and an online demo in LMSYS Chatbot Arena. Stay tuned for our forthcoming code and paper, which will provide more details on the whole process.
*Based on MT Bench evaluations, using GPT-4 scoring. Further human evaluation is needed.
Overview
Supervised fine-tuning (SFT) has demonstrated remarkable effectiveness in developing chatbot systems from language models, particularly when leveraging high-quality data distilled from ChatGPT/GPT-4 (examples include Alpaca, Vicuna, OpenHermes 2.5, and Openchat 3.5). However, the extent to which Reinforcement Learning from Human Feedback (RLHF) or AI feedback (RLAIF) can enhance models when scaling high-quality preference data remains an open question. Earlier endeavors in the open-source community, such as Zephyra-7B, Neural-Chat-7B, and Tulu-2-DPO-70B, employed Direct Preference Optimization (DPO), but their performance in MT Bench (and some in Chatbot Arena), when compared to leading SFT models like OpenHermes 2.5 and Openchat 3.5, has not fully showcased RLHF’s potential.To facilitate more thorough research into RLHF, a high-quality ranking dataset specifically for chat is essential. We release Nectar, a GPT-4 labeled ranking dataset composed of 183K chat prompts. Each prompt includes 7 responses distilled from various models like GPT-4, GPT-3.5-instruct, GPT-3.5-turbo, Mistral-7B-Instruct, Llama2-7B, resulting in a total of 3.8M pairwise comparisons. Considerable effort was invested in mitigating positional bias when prompting GPT-4 for rankings, the details of which are elaborated in the dataset section below.
Moreover, there is a notable scarcity of open-source reward models. We address this gap by releasing our reward model Starling-RM-7B-alpha, trained with our K-wise loss on the Nectar dataset.
Lastly, we fine-tuned the Openchat 3.5 language model using the learned reward model. This resulted in an increase in the MT-Bench score from 7.81 to 8.09, and an improvement in the AlpacaEval score from 88.51% to 91.99%. Both metrics assess the chatbot’s helpfulness.
We hope the open-sourced dataset, reward model and language model can help deepen the understanding of the RLHF mechanism and contribute to AI safety research. Our team is actively exploring various training methodologies for both the reward and language models, and will continue to update this blog with our findings and model releases.
Evaluation of the Model
Evaluating chatbots is never a simple task. We mainly evaluate the helpfulness of our models based on MT-Bench and AlpacaEval, which are GPT-4-based comparisons. We also test the basic capability of the model via MMLU. The results are listed below.In line with findings in GPT-4 Technical Report, our observations post-RLHF reveal similar trends. We’ve observed improvements in the model’s helpfulness and safety features; however, its basic capabilities in areas like knowledge-based QA, math, and coding have either remained static or experienced minor regression. We also detected a tendency for the model to respond with excessive caution to certain benign prompts after initial RLHF, while still showing vulnerabilities to jailbreaking attempts. This may require further fine-tuning with rule-based reward models with GPT-4 as classifiers, similar to what is done in the GPT-4 Technical Report. In the upcoming release of the paper, we will also benchmark the quality of the reward model, and the safety of the language model.
GPT-4's potential in shaping the future of radiology
As the benefits of AI in healthcare continue to emerge, we investigate GPT-4's potential in radiology. Learn about research exploring GPT-4’s potential in assisting report structuring, classifying diseases, and generating comprehensive findings summaries:
www.microsoft.com
GPT-4’s potential in shaping the future of radiology
Published November 27, 2023By Javier Alvarez-Valle , Senior Director of Biomedical Imaging Matthew Lungren , Chief Medical Information Officer, Nuance Communications
Share this page
This research paper is being presented at the 2023 Conference on Empirical Methods in Natural Language Processing(opens in new tab) (EMNLP 2023), the premier conference on natural language processing and artificial intelligence.
- PROJECTProject MAIRA
In recent years, AI has been increasingly integrated into healthcare, bringing about new areas of focus and priority, such as diagnostics, treatment planning, patient engagement. While AI’s contribution in certain fields like image analysis and drug interaction is widely recognized, its potential in natural language tasks with these newer areas presents an intriguing research opportunity.
One notable advancement in this area involves GPT-4’s impressive performance(opens in new tab) on medical competency exams and benchmark datasets. GPT-4 has also demonstrated potential utility(opens in new tab) in medical consultations, providing a promising outlook for healthcare innovation.
Progressing radiology AI for real problems
Our paper, “Exploring the Boundaries of GPT-4 in Radiology(opens in new tab),” which we are presenting at EMNLP 2023(opens in new tab), further explores GPT-4’s potential in healthcare, focusing on its abilities and limitations in radiology—a field that is crucial in disease diagnosis and treatment through imaging technologies like x-rays, computed tomography (CT) and magnetic resonance imaging (MRI). We collaborated with our colleagues at Nuance(opens in new tab), a Microsoft company, whose solution, PowerScribe, is used by more than 80 percent of US radiologists. Together, we aimed to better understand technology’s impact on radiologists’ workflow.Our research included a comprehensive evaluation and error analysis framework to rigorously assess GPT-4’s ability to process radiology reports, including common language understanding and generation tasks in radiology, such as disease classification and findings summarization. This framework was developed in collaboration with a board-certified radiologist to tackle more intricate and challenging real-world scenarios in radiology and move beyond mere metric scores.
We also explored various effective zero-, few-shot, and chain-of-thought (CoT) prompting techniques for GPT-4 across different radiology tasks and experimented with approaches to improve the reliability of GPT-4 outputs. For each task, GPT-4 performance was benchmarked against prior GPT-3.5 models and respective state-of-the-art radiology models.
We found that GPT-4 demonstrates new state-of-the-art performance in some tasks, achieving about a 10-percent absolute improvement over existing models, as shown in Table 1. Surprisingly, we found radiology report summaries generated by GPT-4 to be comparable and, in some cases, even preferred over those written by experienced radiologists, with one example illustrated in Table 2.
Another encouraging prospect for GPT-4 is its ability to automatically structure radiology reports, as schematically illustrated in Figure 1. These reports, based on a radiologist’s interpretation of medical images like x-rays and include patients’ clinical history, are often complex and unstructured, making them difficult to interpret. Research shows that structuring these reports can improve standardization and consistency in disease descriptions, making them easier to interpret by other healthcare providers and more easily searchable for research and quality improvement initiatives. Additionally, using GPT-4 to structure and standardize radiology reports can further support efforts to augment real-world data (RWD) and its use for real-world evidence (RWE). This can complement more robust and comprehensive clinical trials and, in turn, accelerate the application of research findings into clinical practice.
Beyond radiology, GPT-4’s potential extends to translating medical reports into more empathetic(opens in new tab) and understandable formats for patients and other health professionals. This innovation could revolutionize patient engagement and education, making it easier for them and their carers to actively participate in their healthcare.
MICROSOFT RESEARCH PODCAST
Abstracts: October 23, 2023
On “Abstracts,” Partner Research Manager Andy Gordon & Senior Researcher Carina Negreanu explore new work introducing co-audit, a term for any tool-assisted experience that helps users of generative AI find and fix mistakes in AI output.Listen now
Opens in a new tab
A promising path toward advancing radiology and beyond
When used with human oversight, GPT-4 also has the potential to transform radiology by assisting professionals in their day-to-day tasks. As we continue to explore this cutting-edge technology, there is great promise in improving our evaluation results of GPT-4 by investigating how it can be verified more thoroughly and finding ways to improve its accuracy and reliability.Our research highlights GPT-4’s potential in advancing radiology and other medical specialties, and while our results are encouraging, they require further validation through extensive research and clinical trials. Nonetheless, the emergence of GPT-4 heralds an exciting future for radiology. It will take the entire medical community working alongside other stakeholders in technology and policy to determine the appropriate use of these tools and responsibly realize the opportunity to transform healthcare. We eagerly anticipate its transformative impact towards improving patient care and safety.
Learn more about this work by visiting the Project MAIRA(opens in new tab) (Multimodal AI for Radiology Applications) page.
Acknowledgements
We’d like to thank our coauthors: Qianchu Liu, Stephanie Hyland, Shruthi Bannur, Kenza Bouzid, Daniel C. Castro, Maria Teodora Wetscherek, Robert Tinn, Harshyta Sharma, Fernando Perez-Garcia, Anton Schwaighofer, Pranav Rajpurkar, Sameer Tajdin Khanna, Hoifung Poon, Naoto Usuyama, Anja Thieme, Aditya V. Nori, Ozan Oktay
Last edited:
Amazon Introduces Q, an A.I. Chatbot for Companies
Amazon has been racing to shake off the perception that it is lagging in the push to take advantage of artificial intelligence.
Amazon Introduces Q, an A.I. Chatbot for Companies
Amazon has been racing to shake off the perception that it is lagging in the push to take advantage of artificial intelligence.
Amazon’s new A.I. chatbot, Q.CreditCredit...Video by Amazon Web Services
By Karen Weise
Reporting from Seattle
Nov. 28, 2023Updated 2:10 p.m. ET
OpenAI has ChatGPT. Google has the Bard chatbot. Microsoft has its Copilots. On Tuesday, Amazon joined the chatbot race and announced an artificial intelligence assistant of its own: Amazon Q.
The chatbot, developed by Amazon’s cloud computing division, is focused on workplaces and not intended for consumers. Amazon Q aims to help employees with daily tasks, such as summarizing strategy documents, filling out internal support tickets and answering questions about company policy. It will compete with other corporate chatbots, including Copilot, Google’s Duet AI and ChatGPT Enterprise.
“We think Q has the potential to become a work companion for millions and millions of people in their work life,” Adam Selipsky, the chief executive of Amazon Web Services, said in an interview.
Image
Adam Selipsky, the head of Amazon Web Services, last year. He said companies wanted to use chatbots but were concerned about data security and privacy.Credit...Noah Berger/Amazon Web Services, via Associated Press
Amazon has been racing to shake off the perception that it is lagging behind in the A.I. competition. In the year since OpenAI released ChatGPT, Google, Microsoft and others have jumped into the frenzy by unveiling their own chatbots and investing heavily in A.I. development.
Amazon was quieter about its A.I. plans until more recently. In September, it announced that it would invest up to $4 billion in Anthropic, an A.I. start-up that competes with OpenAI, and develop advanced computing chips together. Amazon also introduced a platform this year that allows customers to have access to different A.I. systems.
As the leading provider of cloud computing, Amazon already has business customers storing vast amounts of information on its cloud servers. Companies were interested in using chatbots in their workplaces, Mr. Selipsky said, but they wanted to make sure the assistants would safeguard those hoards of corporate data and keep their information private.
Many companies “told me that they had banned these A.I. assistants from the enterprise because of the security and privacy concerns,” he said.
In response, Amazon built Q to be more secure and private than a consumer chatbot, Mr. Selipsky said. Amazon Q, for example, can have the same security permissions that business customers have already set up for their users. At a company where an employee in marketing may not have access to sensitive financial forecasts, Q can emulate that by not providing that employee with such financial data when asked.
Image
Amazon Q is intended to help employees with daily tasks, including answering questions about corporate policy.Credit...Amazon Web Services
Credit...Amazon Web Services
Companies can also give Amazon Q permission to work with their corporate data that isn’t on Amazon’s servers, such as connecting with Slack and Gmail.
Unlike ChatGPT and Bard, Amazon Q is not built on a specific A.I. model. Instead, it uses an Amazon platform known as Bedrock, which connects several A.I. systems together, including Amazon’s own Titan as well as ones developed by Anthropic and Meta.
The name Q is a play on the word “question,” given the chatbot’s conversational nature, Mr. Selipsky said. It is also a play on the character Q in the James Bond novels, who makes stealthy, helpful tools, and on a powerful “Star Trek” figure, he added.
Pricing for Amazon Q starts at $20 per user each month. Microsoft and Google both charge $30 a month for each user of the enterprise chatbots that work with their email and other productivity applications.
Amazon Q was one of a slew of announcements that the company made at its annual cloud computing conference in Las Vegas. It also shared plans to beef up its computing infrastructure for A.I. And it expanded a longtime partnership with Nvidia, the dominant supplier of A.I. chips, including by building what the companies called the world’s fastest A.I. supercomputer.
Image
An image of Nvidia and Amazon’s forthcoming DGX Cloud Project Ceiba, which they describe as the world’s fastest A.I. supercomputer.Credit...via Nvidia
Most such systems use standard microprocessors along with specialized chips from Nvidia called GPUs, or graphics processing units. Instead, the system announced on Tuesday will be built with new Nvidia chips that include processor technology from Arm, the company whose technology powers most mobile phones.
Image
An image of Nvidia’s GH200 Grace Hopper A.I. Superchip, which the supercomputer will use.Credit...via Nvidia
The shift is a troubling sign for Intel and Advanced Micro Devices, the dominant microprocessor suppliers. But it is positive news for Arm in its long-running effort to break into data center computers.
Don Clark contributed reporting from San Francisco.
Putin: West dominating AI industry, Russia must step up
Putin says foreign LLMs are biased and ignore the Russian culture.%2Favatars%2FEwuSxftyWaVt.png&w=96&q=75 "Sejal Sharma")
Sejal Sharma
Published: Nov 27, 2023 11:06 AM EST
CULTURE
/2023/11/27/image/jpeg/7DGD9w05UB9DvtonUzPZemlhWUC1C0p6lfpNHePo.jpg "Russian President Vladimir Putin")
Russian President Vladimir Putin
Wikimedia Commons
Get a daily digest of the latest news in tech, science, and technology, delivered right to your mailbox. Subscribe now.
By subscribing, you agree to our Terms of Use and Policies You may unsubscribe at any time.In a call for domestic artificial intelligence models that reflect Russian culture and are trained on Russia-specific data, President Vladimir Putin said that “monopolistic dominance” of foreign AI models is unacceptable and dangerous.
Putin was speaking at the Artificial Intelligence Journey 2023 international AI and machine learning conference, which took place in Moscow on Friday.
Listen to his full speech here:
Staking claim in the AI arms race
AI has become the central point of contention in the arms race between the United States and China, the most dominant countries in the development of the technology.Last month, The Biden administration imposed more bans on importing US-produced AI chips to China. These bans are meant to inhibit China’s advancements from gaining technological advantage.
In the global dance of technological aspirations, Putin doubled on Moscow’s dreams of waltzing into the realm of AI supremacy.
However, the war in Ukraine has thrown a wrench into those plans. The conflict has caused an exodus of talent from the country and further pressure mounting after sanctions by the West, putting the brakes on its high-tech imports.
Putin acknowledged the turbulence at the conference. Despite the occasionally disconcerting ethical and social repercussions of emerging technologies, Putin declared that banning AI wasn’t an option.
Since it exploded on the scene last year, OpenAI’s ChatGPT has been banned in Russia.
New strategy for AI development
Putin announced that he’s about to nod to a fresh version of Russia's game plan for AI development. He's throwing down the gauntlet and announced that a new AI advancement bill will be signed. He has also called for a major boost in the number of supercomputers."This is critically necessary for the further development of generative artificial intelligence… In the very near future, as one of the first steps, a presidential decree will be signed and a new version of the national strategy for the development of artificial intelligence will be approved," Putin said at the conference.
Coming down on large language models that are “biased” as they have been trained in English, Putin said that these models sometimes ignore or cancel Russian culture, equivalent to “a kind of abolition in the digital space.”
According to Stanford’s Artificial Intelligence Index Report 2023, American institutions produce most of the world’s large language and multimodal models (54 percent in 2022).
In a call to end their dependency on the West for technology, Putin said, “How can we make our country more efficient and ensure the happiness of all people? This is a question for artificial intelligence.”
There's never a dull moment when it comes to A.I.
"AI tool GNoME finds 2.2 million new crystals, including 380,000 stable materials that could power future technologies
Modern technologies from computer chips and batteries to solar panels rely on inorganic crystals. To enable new technologies, crystals must be stable otherwise they can decompose, and behind each new, stable crystal can be months of painstaking experimentation.
Today, in a paper published in Nature, we share the discovery of 2.2 million new crystals – equivalent to nearly 800 years’ worth of knowledge. We introduce Graph Networks for Materials Exploration (GNoME), our new deep learning tool that dramatically increases the speed and efficiency of discovery by predicting the stability of new materials.
With GNoME, we’ve multiplied the number of technologically viable materials known to humanity. Of its 2.2 million predictions, 380,000 are the most stable, making them promising candidates for experimental synthesis. Among these candidates are materials that have the potential to develop future transformative technologies ranging from superconductors, powering supercomputers, and next-generation batteries to boost the efficiency of electric vehicles.
GNoME shows the potential of using AI to discover and develop new materials at scale. External researchers in labs around the world have independently created 736 of these new structures experimentally in concurrent work. In partnership with Google DeepMind, a team of researchers at the Lawrence Berkeley National Laboratory has also published a second paper in Nature that shows how our AI predictions can be leveraged for autonomous material synthesis.
We’ve made GNoME’s predictions available to the research community. We will be contributing 380,000 materials that we predict to be stable to the Materials Project, which is now processing the compounds and adding them into its online database. We hope these resources will drive forward research into inorganic crystals, and unlock the promise of machine learning tools as guides for experimentation"
deepmind.google
ChatGPT4's response:
The study "Scaling deep learning for materials discovery," led by Amil Merchant, Simon Batzner, Samuel S. Schoenholz, Muratahan Aykol, Gowoon Cheon, and Ekin Dogus Cubuk, explores the use of deep learning, specifically Graph Neural Networks (GNNs), to accelerate the discovery of new materials. Here's a breakdown of their experiment and the implications of their discovery:
Millions of new materials discovered with deep learning
"AI tool GNoME finds 2.2 million new crystals, including 380,000 stable materials that could power future technologies
Modern technologies from computer chips and batteries to solar panels rely on inorganic crystals. To enable new technologies, crystals must be stable otherwise they can decompose, and behind each new, stable crystal can be months of painstaking experimentation.
Today, in a paper published in Nature, we share the discovery of 2.2 million new crystals – equivalent to nearly 800 years’ worth of knowledge. We introduce Graph Networks for Materials Exploration (GNoME), our new deep learning tool that dramatically increases the speed and efficiency of discovery by predicting the stability of new materials.
With GNoME, we’ve multiplied the number of technologically viable materials known to humanity. Of its 2.2 million predictions, 380,000 are the most stable, making them promising candidates for experimental synthesis. Among these candidates are materials that have the potential to develop future transformative technologies ranging from superconductors, powering supercomputers, and next-generation batteries to boost the efficiency of electric vehicles.
GNoME shows the potential of using AI to discover and develop new materials at scale. External researchers in labs around the world have independently created 736 of these new structures experimentally in concurrent work. In partnership with Google DeepMind, a team of researchers at the Lawrence Berkeley National Laboratory has also published a second paper in Nature that shows how our AI predictions can be leveraged for autonomous material synthesis.
We’ve made GNoME’s predictions available to the research community. We will be contributing 380,000 materials that we predict to be stable to the Materials Project, which is now processing the compounds and adding them into its online database. We hope these resources will drive forward research into inorganic crystals, and unlock the promise of machine learning tools as guides for experimentation"
Millions of new materials discovered with deep learning
We share the discovery of 2.2 million new crystals – equivalent to nearly 800 years’ worth of knowledge. We introduce Graph Networks for Materials Exploration (GNoME), our new deep learning tool...
ChatGPT4's response:
The study "Scaling deep learning for materials discovery," led by Amil Merchant, Simon Batzner, Samuel S. Schoenholz, Muratahan Aykol, Gowoon Cheon, and Ekin Dogus Cubuk, explores the use of deep learning, specifically Graph Neural Networks (GNNs), to accelerate the discovery of new materials. Here's a breakdown of their experiment and the implications of their discovery:
Experiment Overview:
- Objective: To improve the efficiency of materials discovery using deep learning. The team focused on inorganic crystals, vital for various technological applications.
- Methodology:
- Data Gathering: The researchers started with a database of 48,000 known stable crystals.
- Deep Learning Model: They employed Graph Networks for Materials Exploration (GNoME), a type of GNN, trained on this vast dataset.
- Active Learning: The model was iteratively trained and used to filter candidate structures. The energy of these candidates was computed using Density Functional Theory (DFT), a quantum mechanical modelling method.
- Discovery and Validation: Through this process, the model discovered over 2.2 million new structures. 736 of these were later validated through independent experimental realization.
- Tools Used:
- DFT Calculations: For accurate energy estimations of materials.
- GNNs: To predict the total energy of a crystal from its structure and composition.
- Data from Existing Databases: Like the Materials Project and Open Quantum Materials Database (OQMD).
Key Findings:
- Massive Expansion of Known Stable Materials: The study led to the discovery of 2.2 million new structures, significantly expanding the catalog of stable materials known to humanity.
- Efficiency in Discovery: Their approach was orders of magnitude more efficient than traditional trial-and-error methods.
- Model Generalization: The GNoME model demonstrated strong generalization capabilities, accurately predicting structures even with more than five unique elements, which was previously challenging.
- Learned Interatomic Potentials: The scale and diversity of calculations enabled the development of highly accurate and robust interatomic potentials for molecular-dynamics simulations.
Implications:
- Acceleration of Material Discovery: This approach can significantly speed up the discovery of new materials, essential for various applications like clean energy, batteries, and photovoltaics.
- Reduced Reliance on Trial-and-Error: By leveraging machine learning, the need for expensive and time-consuming experimental approaches can be reduced.
- Broader Impact on Science and Technology: The ability to quickly discover new materials could lead to breakthroughs in several fields, from electronics to energy storage.
- Potential for Future Discoveries: The study demonstrates the vast potential of machine learning in materials science, indicating that similar approaches could lead to even more discoveries.
- Challenges in Material Synthesis: While the discovery of new materials is accelerated, the challenge of synthesizing these materials in the lab remains.
Last edited: