The A.I Megathread (LLM , GPT , Development)

[aXiom]

Well, A.I progress was because of the 8 years of work OpenAI was doing previously. All those plugins and everything they're releasing easy now were built on years of work.

Now, CS is a life-long learning field. Nobody learns everything in 4 years of college. However, getting that CS Degree means you don't have to answer any questions about why you don't have a degree for 50+ years or however long your career is.

I said it before the layoffs: you don't need a degree to thrive in this field as long as you learn and can show you can do the job. Bootcamp, self-taught, etc. all no problems.

However now? Every single thing helps. Additionally, if you're going to grind through algo, stats, LA, etc. you might as well get credit for it. shyt, many schools and curriculums like mine had an A.I course. Additionally, courses like compilers, OS, architecture, etc. are there if somebody wants to step away from Full Stack and go into embedded or hardware. Not to mention some places offer courses about how to write scalable, readable code, which is absolutely critical for teams.

Especially in fields that are math heavy. If you got no experience and are applying for a job over a U.C Math or CS graduate, it's going to be tough.

Those 10 subjects mentioned are quite stacked. I got a CS degree, been working in the field for about 5 years and I don't know all those topics. I'm going to say that's roughly the same for the average person as well. shyt, somebody at work last week was struggling with Git.

The bolded is true. I'll also add that most of the progress that OpenAI made over the past few years was built on top of work doen by Google.

That being said, it's one thing to know how to write code, it's another thing to understand how all these systems integrate and work in unison to provide a service or product. Tech is constantly a moving target where the most important attributes for career progress are curiosity and problem solving. Everything else will fall into play. Too many times I've seen devs fukk shyt up and point fingers at everyone else while costing companies thousands to hundreds of thousand in lost man hours and revenue because they're one dimensional and they have no interest in learning where their responsibilities end and others' begin.

I'm not even saying that degrees hold no weight, they do, but tech is a field in which the demand of good engineers outstrips supply and while we're seeing flashes of that changing in the future, it's not as close as some may think. A.I. tooling will only widen the gap between good engineers and mediocre ones, so expect pain at the entry level, and expect the below average performing people to get hit first. To the rest, it's just a force multiplier. By the time it affects the ones that actually make shyt happen the world will either have much bigger issues or they'll already have been solved.

 

[IIVI]

The bolded is true. I'll also add that most of the progress that OpenAI made over the past few years was built on top of work doen by Google.

That being said, it's one thing to know how to write code, it's another thing to understand how all these systems integrate and work in unison to provide a service or product. Tech is constantly a moving target where the most important attributes for career progress are curiosity and problem solving. Everything else will fall into play. Too many times I've seen devs fukk shyt up and point fingers at everyone else while costing companies thousands to hundreds of thousand in lost man hours and revenue because they're one dimensional and they have no interest in learning where their responsibilities end and others' begin.

I'm not even saying that degrees hold no weight, they do, but tech is a field in which the demand of good engineers outstrips supply and while we're seeing flashes of that changing in the future, it's not as close at some may think. A.I. tooling will only widen the gap between good engineers and mediocre ones, so expect pain at the entry level, and expect the below average performing people to get hit first. To the rest, it's just a force multiplier. By the time it affects the ones that actually make shyt happen the world will either have much bigger issues or they'll already been solved.

Well said. I think if you've already got your foot in the door and have years to show, you're good. Degree or not.

My main gripe with the initial tweet is it had that 2019-2022 vibe where a lot of people were saying "It's easy to get into tech, learn this and start and making $300k/year!" Like you mentioned, especially right now the entry level is highly competitive - we're talking about the best people around the world are fighting for jobs.

While A.I is not my area of expertise, I do think what was mentioned was a solid list and people who understand those subjects are most likely going to know their stuff, work well in the industry and be a positive to their teams. However, for an entry level person being able to convince an employer they can do it in this climate is the difficult thing. People who are going in as a self-taught programmers will have to compete with the CS/STEM degree holders and Bootcampers. When people were hiring left and right, no problems. However now with smaller teams, seniors on the job are busy and they don't got time to interview every single person so they're going to focus on the ones with the accolades on those resumes.

 

[bnew]

 

[bnew]

GitHub - wawawario2/text-generation-webui: A gradio web UI for running Large Language Models like GPT-J 6B, OPT, GALACTICA, LLaMA, and Pygmalion.

A gradio web UI for running Large Language Models like GPT-J 6B, OPT, GALACTICA, LLaMA, and Pygmalion. - GitHub - wawawario2/text-generation-webui: A gradio web UI for running Large Language Models...

github.com

Text Generation Web UI with Long-Term Memory​

Welcome to the experimental repository for the Text Generation Web UI with a long-term memory (LTM) extension. The goal of the LTM extension is to enable the chatbot to "remember" conversations long-term. Please note that this is an early-stage experimental project, and perfect results should not be expected. This project has been tested on Ubuntu LTS 22.04. Other people have tested it successfully on Windows. Compatibility with macOS is unknown.

 

[bnew]

Spoiler: larger images

https://archive.is/GFWbX

 

[bnew]

Spoiler: larger image

https://archive.is/FLKYT

 

[bnew]

https://archive.is/x46qo

Spoiler: larger image

 

[bnew]

YouTuber teaches ChatGPT how to crack Windows 95 keys

A YouTuber, who goes by the handle Enderman, managed to get ChatGPT to create valid Windows 95 activation codes. He initially just asked the bot outright to...

www.techspot.com

 

[spliz]

This whole AI shyt makes me extremely uncomfortable and triggers my anxiety I can't front. Lol

 

[bnew]

Bloomberg - Are you a robot?

Introducing BloombergGPT, Bloomberg’s 50-billion parameter large language model, purpose-built from scratch for finance​

March 30, 2023

BloombergGPT outperforms similarly-sized open models on financial NLP tasks by significant margins — without sacrificing performance on general LLM benchmarks​

NEW YORK – Bloomberg today released a research paper detailing the development of BloombergGPTTM, a new large-scale generative artificial intelligence (AI) model. This large language model (LLM) has been specifically trained on a wide range of financial data to support a diverse set of natural language processing (NLP) tasks within the financial industry.

Recent advances in Artificial Intelligence (AI) based on LLMs have already demonstrated exciting new applications for many domains. However, the complexity and unique terminology of the financial domain warrant a domain-specific model. BloombergGPT represents the first step in the development and application of this new technology for the financial industry. This model will assist Bloomberg in improving existing financial NLP tasks, such as sentiment analysis, named entity recognition, news classification, and question answering, among others. Furthermore, BloombergGPT will unlock new opportunities for marshalling the vast quantities of data available on the Bloomberg Terminal to better help the firm’s customers, while bringing the full potential of AI to the financial domain.

For more than a decade, Bloomberg has been a trailblazer in its application of AI, Machine Learning, and NLP in finance. Today, Bloomberg supports a very large and diverse set of NLP tasks that will benefit from a new finance-aware language model. Bloomberg researchers pioneered a mixed approach that combines both finance data with general-purpose datasets to train a model that achieves best-in-class results on financial benchmarks, while also maintaining competitive performance on general-purpose LLM benchmarks.

To achieve this milestone, Bloomberg’s ML Product and Research group collaborated with the firm’s AI Engineering team to construct one of the largest domain-specific datasets yet, drawing on the company’s existing data creation, collection, and curation resources. As a financial data company, Bloomberg’s data analysts have collected and maintained financial language documents over the span of forty years. The team pulled from this extensive archive of financial data to create a comprehensive 363 billion token dataset consisting of English financial documents.

This data was augmented with a 345 billion token public dataset to create a large training corpus with over 700 billion tokens. Using a portion of this training corpus, the team trained a 50-billion parameter decoder-only causal language model. The resulting model was validated on existing finance-specific NLP benchmarks, a suite of Bloomberg internal benchmarks, and broad categories of general-purpose NLP tasks from popular benchmarks (e.g., BIG-bench Hard, Knowledge Assessments, Reading Comprehension, and Linguistic Tasks). Notably, the BloombergGPT model outperforms existing open models of a similar size on financial tasks by large margins, while still performing on par or better on general NLP benchmarks.

Table 1. How BloombergGPT performs across two broad categories of NLP tasks: finance-specific and general-purpose.


“For all the reasons generative LLMs are attractive – few-shot learning, text generation, conversational systems, etc. – we see tremendous value in having developed the first LLM focused on the financial domain,” said Shawn Edwards, Bloomberg’s Chief Technology Officer. “BloombergGPT will enable us to tackle many new types of applications, while it delivers much higher performance out-of-the-box than custom models for each application, at a faster time-to-market.”

“The quality of machine learning and NLP models comes down to the data you put into them,” explained Gideon Mann, Head of Bloomberg’s ML Product and Research team. “Thanks to the collection of financial documents Bloomberg has curated over four decades, we were able to carefully create a large and clean, domain-specific dataset to train a LLM that is best suited for financial use cases. We’re excited to use BloombergGPT to improve existing NLP workflows, while also imagining new ways to put this model to work to delight our customers.”

For more details about the development of BloombergGPT, read the paper on arXiv: https://arxiv.org/abs/2303.17564.

 

[bnew]

google bard explanation of a Token.

In the context of AI models, a token is a small piece of data that is used to represent a word, phrase, or other unit of meaning. Tokens are created by breaking down text into smaller units, such as words, characters, or subwords.

For example, the sentence "I love you" could be tokenized into the following tokens:

• I
• love
• you

Tokens are then used to represent the input to an AI model. The model then processes the tokens and generates an output, such as a translation, a summary, or a prediction.

Tokens are an important part of AI models because they allow the model to represent the meaning of text in a way that is understandable by the model. This is necessary for the model to be able to process and understand text, and to generate meaningful output.

There are a number of different ways to tokenize text. The most common way is to use a dictionary to map words to tokens. However, other methods can also be used, such as using a rule-based system or a statistical model.

The choice of tokenization method will depend on the specific AI model that is being used. Some models require a specific type of tokenization, while others are more flexible.

 

[bnew]

Spoiler: larger image

 

[bnew]

https://archive.is/Owwc9

Markprompt

Open-source GPT-4 platform for Markdown, Markdoc and MDX with built-in analytics

markprompt.com

 

[bnew]

https://archive.is/nXe7D

 

[bnew]

https://archive.is/HUh8r

GitHub - turboncelucionce/gpt4-clippy-godot

Contribute to turboncelucionce/gpt4-clippy-godot development by creating an account on GitHub.

github.com

GPT-4 (or 3.5)-Powered Clippy​

Overview​

This is a little godot project that makes Microsoft Clippy appear on your desktop. His responses are fully generated by GPT-3.5 or 4. Current features:

• You can type, or directly talk to Clippy using your microphone. The audio is automatically transcribed using OpenAI's Whisper API.
• Clippy can talk realistically using any voice you want by providing an ElevenLabs API key and your character voices.
• Clippy reacts to the user's input, like the user hovering their mouse over him, sending blank messages, being ignored for long periods of time, or clicking on him repeatedly. (In the code, these are referred to as "nonverbal" actions.)
• Clippy can memorize things between conversations.
• Clippy can be dragged across the screen and set to be Always On Top.