Y'all heard about ChatGPT yet? AI instantly generates question answers, entire essays etc.
- Thread starter RamsayBolton
- Start date
Sensitive Christian Grey
The Fisher King
cyndaquil
Lv 100 Bold natured
Hopefully not. We still need stack overflow for certain questions. Probably just means better questions will be asked there instead of some of the trivial shyt people ask like "how to reverse a string if given a pointer to the index of the first character?"
1/2
@pedrodias
Get rid of extensions that do "everything" and replace their core functionalities that you really need with JS bookmarklets.
You will get a less bloated Chrome, with less interference in your testing needs. Plus you'll get more privacy and less exposure to extensions gone rogue
2/2
@pedrodias
If you have no coding skills, it's easier than ever to get a JS bookmarklet using ChatGPT or Claude
To post tweets in this format, more info here: https://www.thecoli.com/threads/tips-and-tricks-for-posting-the-coli-megathread.984734/post-52211196
@pedrodias
Get rid of extensions that do "everything" and replace their core functionalities that you really need with JS bookmarklets.
You will get a less bloated Chrome, with less interference in your testing needs. Plus you'll get more privacy and less exposure to extensions gone rogue
2/2
@pedrodias
If you have no coding skills, it's easier than ever to get a JS bookmarklet using ChatGPT or Claude
To post tweets in this format, more info here: https://www.thecoli.com/threads/tips-and-tricks-for-posting-the-coli-megathread.984734/post-52211196
1/3
@SullyOmarr
my only question about search gpt:
how on earth is it so fast?
seriously, what and how did they cook, its nearly instant
2/3
@BorisMPower
Early feedback was that it’s slow, so we put effort in making it super fast
3/3
@3DX3EM
Will it allow pass search query as a URL parameter i.e. like CoPilot does? so I can make my papers search bookmarklet to use ChatGPT search
To post tweets in this format, more info here: https://www.thecoli.com/threads/tips-and-tricks-for-posting-the-coli-megathread.984734/post-52211196
@SullyOmarr
my only question about search gpt:
how on earth is it so fast?
seriously, what and how did they cook, its nearly instant
2/3
@BorisMPower
Early feedback was that it’s slow, so we put effort in making it super fast
3/3
@3DX3EM
Will it allow pass search query as a URL parameter i.e. like CoPilot does? so I can make my papers search bookmarklet to use ChatGPT search
To post tweets in this format, more info here: https://www.thecoli.com/threads/tips-and-tricks-for-posting-the-coli-megathread.984734/post-52211196
Illuminatos
#OVOXO
Late pass: I had no idea this thing could read and analyze text on png files. Lmao this is insane. So many people are dooming and glooming about AI and everyday my mind is constantly blown at how smart this thing is. Absolutely amazing tech.
Late pass: I had no idea this thing could read and analyze text on png files. Lmao this is insane. So many people are dooming and glooming about AI and everyday my mind is constantly blown at how smart this thing is. Absolutely amazing tech.
better late than never but
1/11
@btibor91
Is there a problem with the new ChatGPT-4o model? In this case, it responds with random Python snippets and seemingly completely ignores the prompt
Some of the benchmarks I have seen suggest it has gotten worse for coding, possibly hinting at a smaller model - is anyone else seeing this?
[Quoted tweet]
ChatGPT-4o's Bizarre Coding Response
Had the weirdest experience with ChatGPT-4o today. Asked it to create a simple Angular todo list app, and instead got... a Python script about checking the weather? Yes, really.
I double-checked my prompt to make sure I was clear. But nope – where I should've gotten components, services, and TypeScript code, I got a random Python script with LangChain imports and weather API calls. It's like asking for a burger and getting a tennis racket instead.
The strangest part? Earlier versions handled these kinds of requests just fine. This wasn't just a small mix-up – the model completely missed the mark, repeatedly giving me everything except Angular code.
Anyone else seeing similar coding issues with the latest version? Drop your experiences in the comments below.
Link in comment
2/11
@test_tm7873
running on Q5 bit qualization because of all that sora things
3/11
@AspieJames
Usually happens when they're about to roll something out...
4/11
@sithamet
Not a new behavior: they did it before twice:
1. When their GPU config broke.
2. When they tested a weird caching and took it down after complaints.
OpenAI has a notch for shrinking computing at the cost of performance; we will see how it goes. Haven't used 4o for coding for a while, though.
5/11
@ToonamiAfter
Its either a smaller model or there are new bugs. I've seen it hallucinate much more than previous 4o versions and just break
6/11
@JakubNorkiewicz
OpenAI’s revenge for stealing fire from Machine Gods
7/11
@mahaoo_ASI
I've had several iterations where it ignored my prompt and as if answered a different prompt or sometimes repeated itself from a previous answer in the same chat
I assumed they either have some sort of backend testing of what happens to users when they give an answer that is retrieved from past responses (or just a regular bug) and moved to claude
Overall, I've found gpt4o can no longer be used for coding, it's either o1 series or sonnet
8/11
@Hangsiin
I never use 4o for coding recently. completely useless.
9/11
@techikansh
Yup,
- bad outputs…
- Takes too long to output
Only good for writing emails, translation and minecraft i guess…
Cancelled my subscription today…
Will get back on the wagon if they ship something better…
10/11
@erictronai
There’s something off, haven’t used it enough for code yet, but it doesn’t seem to perform as well across a longer conversation or be able to organise more complex ideas as coherently. But I’m going back and forth and running on vibes at the moment
11/11
@CompilerMech
o1 is also making same. It solves the problems in each possible and general way. They are not think simple. They are acting like over-confused engineer.
To post tweets in this format, more info here: https://www.thecoli.com/threads/tips-and-tricks-for-posting-the-coli-megathread.984734/post-52211196
@btibor91
Is there a problem with the new ChatGPT-4o model? In this case, it responds with random Python snippets and seemingly completely ignores the prompt
Some of the benchmarks I have seen suggest it has gotten worse for coding, possibly hinting at a smaller model - is anyone else seeing this?
[Quoted tweet]
ChatGPT-4o's Bizarre Coding Response
Had the weirdest experience with ChatGPT-4o today. Asked it to create a simple Angular todo list app, and instead got... a Python script about checking the weather? Yes, really.
I double-checked my prompt to make sure I was clear. But nope – where I should've gotten components, services, and TypeScript code, I got a random Python script with LangChain imports and weather API calls. It's like asking for a burger and getting a tennis racket instead.
The strangest part? Earlier versions handled these kinds of requests just fine. This wasn't just a small mix-up – the model completely missed the mark, repeatedly giving me everything except Angular code.
Anyone else seeing similar coding issues with the latest version? Drop your experiences in the comments below.
Link in comment
2/11
@test_tm7873
running on Q5 bit qualization because of all that sora things
3/11
@AspieJames
Usually happens when they're about to roll something out...
4/11
@sithamet
Not a new behavior: they did it before twice:
1. When their GPU config broke.
2. When they tested a weird caching and took it down after complaints.
OpenAI has a notch for shrinking computing at the cost of performance; we will see how it goes. Haven't used 4o for coding for a while, though.
5/11
@ToonamiAfter
Its either a smaller model or there are new bugs. I've seen it hallucinate much more than previous 4o versions and just break
6/11
@JakubNorkiewicz
OpenAI’s revenge for stealing fire from Machine Gods
7/11
@mahaoo_ASI
I've had several iterations where it ignored my prompt and as if answered a different prompt or sometimes repeated itself from a previous answer in the same chat
I assumed they either have some sort of backend testing of what happens to users when they give an answer that is retrieved from past responses (or just a regular bug) and moved to claude
Overall, I've found gpt4o can no longer be used for coding, it's either o1 series or sonnet
8/11
@Hangsiin
I never use 4o for coding recently. completely useless.
9/11
@techikansh
Yup,
- bad outputs…
- Takes too long to output
Only good for writing emails, translation and minecraft i guess…
Cancelled my subscription today…
Will get back on the wagon if they ship something better…
10/11
@erictronai
There’s something off, haven’t used it enough for code yet, but it doesn’t seem to perform as well across a longer conversation or be able to organise more complex ideas as coherently. But I’m going back and forth and running on vibes at the moment
11/11
@CompilerMech
o1 is also making same. It solves the problems in each possible and general way. They are not think simple. They are acting like over-confused engineer.
To post tweets in this format, more info here: https://www.thecoli.com/threads/tips-and-tricks-for-posting-the-coli-megathread.984734/post-52211196
1/3
@rohanpaul_ai
01-preview achieve 95%+ accuracy on CompTIA security tests.
Very tight performance clustering among top models (95.72% to 92.40%) suggests diminishing returns past certain model sizes.
2/3
@antlionai
There is a world of difference between 92 and 95% because at that point it rbcome how many multiple steps can be completed without failure(not quite sure that's what this is measuring).
So i view it more as a 8 to 5% failure rate.
Still clear that models are no longer getting exponentially better, though they may soon become vastly more useful.
3/3
@rohanpaul_ai
To post tweets in this format, more info here: https://www.thecoli.com/threads/tips-and-tricks-for-posting-the-coli-megathread.984734/post-52211196
@rohanpaul_ai
01-preview achieve 95%+ accuracy on CompTIA security tests.
Very tight performance clustering among top models (95.72% to 92.40%) suggests diminishing returns past certain model sizes.
2/3
@antlionai
There is a world of difference between 92 and 95% because at that point it rbcome how many multiple steps can be completed without failure(not quite sure that's what this is measuring).
So i view it more as a 8 to 5% failure rate.
Still clear that models are no longer getting exponentially better, though they may soon become vastly more useful.
3/3
@rohanpaul_ai
To post tweets in this format, more info here: https://www.thecoli.com/threads/tips-and-tricks-for-posting-the-coli-megathread.984734/post-52211196
OpenAI's Sora is launching today — here are highlights from the first review | TechCrunch
Sora, OpenAI's video generator, is launching today — at least for some users. YouTuber Marques Brownlee got early access.
techcrunch.com
OpenAI’s Sora is launching today — here are highlights from the first review
Kyle Wiggers
9:24 AM PST · December 9, 2024
https://techcrunch.com/2024/12/09/o...es-highlights-from-the-first-review/#comments
Sora, OpenAI’s video generator, is launching Monday — at least for some users.
YouTuber Marques Brownlee revealed the news in a video published to his channel this morning. Brownlee got early access to Sora, and gave his initial impressions in a 15-minute review.
Sora lives on Sora.com, Brownlee said, the homepage for which shows a scroll of recently generated and OpenAI-curated Sora videos. (It hadn’t gone live for us here at TechCrunch as of publication time.) Notably, the tool isn’t built into ChatGPT, OpenAI’s AI-powered chatbot platform. Sora seems to be its own separate experience for now.
Videos on the Sora homepage can be bookmarked for later viewing to a “Saved” tab, organized into folders, and clicked on to see which text prompts were used to create them. Sora can generate videos from uploaded images as well as prompts, according to Brownlee, and can edit existing Sora-originated videos.
Using the “Re-mix” feature, users can describe changes they want to see in a video and Sora will attempt to incorporate these in a newly generated clip. Re-mix has a “strength” setting that lets users specify how drastically they want Sora to change the target video, with higher values yielding videos that take more artistic liberties.
Sora can generate up to 1080p footage, Brownlee said — but the higher the resolution, the longer videos take to generate. 1080p footage takes 8x longer than 480p, the fastest option, while 720p takes 4x longer.
Brownlee said that the average 1080p video took a “couple of minutes” to generate in his testing. “That’s also, like, right now, when almost no one else is using it,” he said. “I kind of wonder how much longer it’ll take when this is just open for anyone to use.”
In addition to generating one-off clips, Sora has a “Storyboard” feature that lets users string together prompts to create scenes or sequences of videos, Brownlee said. This is meant to help with consistency, presumably — a notorious weak point for AI video generators.
But how does Sora perform? Well, Brownlee said, it suffers from the same flaws as other generative tools out there, namely issues related to object permanence. In Sora videos, objects pass in front of each other or behind each other in ways that don’t make sense, and disappear and reappear without any reason.
Legs are another major source of problems for Sora, Brownlee said. Any time a person or animal with legs has to walk for a long while in a clip, Sora will confuse the front legs and back legs. The legs will “swap” back and forth in an anatomically impossible way, Brownlee said.
Sora has a number of safeguards built in, Brownlee said, and prohibits creators from generating footage showing people under the age of 18, containing violence or “explicit themes,” and that might infringe on a third party’s copyright. Sora also won’t generate videos from images with public figures, recognizable characters, or logos, Brownlee said, and it watermarks each video — albeit with a visual watermark that can be easily cropped out.
So, what’s Sora good for? Brownlee found it to be useful for things like title slides in a certain style, animations, abstracts, and stop-motion footage. But he stopped short of endorsing it for anything photorealistic.
“It’s impressive that it’s AI-generated video, but you can tell pretty quickly that it’s AI-generated video,” he said of the majority of Sora’s clips. “Things just get really wonky.”
Ethan Mollick (@emollick.bsky.social)
I have now had multiple instances where professors have told me that they are pretty sure that o1 found something novel in their field, but that the results are technical, non-obvious and complex enough that they can't be sure without considerable checking. An interesting problem.
bsky.app
1/18
Ethan Mollick
I have now had multiple instances where professors have told me that they are pretty sure that o1 found something novel in their field, but that the results are technical, non-obvious and complex enough that they can't be sure without considerable checking. An interesting problem.
2/18
Ethan Mollick @emollick.bsky.social
To be clear, these aren’t earth-shattering discoveries. You can’t ask for “a cure for cancer” or “a new form of physics” and get anything real. These are narrow problems in fields like economics and computer science, and require experts to identify and frame them.
3/18
Drew Curtis @drewcurtis.bsky.social
still important
4/18
猫好きな人 @valoisdubins.bsky.social
What sorts of fields?
5/18
Ethan Mollick @emollick.bsky.social
economics and computer science
6/18
猫好きな人 @valoisdubins.bsky.social
Are these theoretical innovations? Or analyses of data?
7/18
jalex32x.bsky.social @jalex32x.bsky.social
What is the significance of finding something novel independent of it being something important? My Roomba can reach places I don’t bother to vacuum. Can o1 solve alpha fold-like problems?
8/18
Lukas Bergstrom @lukas.blue
That's interesting, because my heuristic for when AI is useful is whenever generating an answer is expensive and checking it is easy.
9/18
Ethan Mollick @emollick.bsky.social
It was until o1
10/18
Lukas Bergstrom @lukas.blue
I'm not sure o1's accuracy has improved enough to make the ROI positive for expensive-to-check answers. If it has, that would be an ever bigger deal than o1's ability to tackle bigger, harder problems.
11/18
thebaldcoder.bsky.social @thebaldcoder.bsky.social
Regardless of using AI or not I'd expect you to check the answer or in software parlance "debug it". AI is just saving you the first step of generating the answer from scratch: you near instantly have something to work with. That alone is massive.
12/18
kiteet.bsky.social @kiteet.bsky.social
Really the only way to verify anything is to do tests with specific samples under specific circumstances. I haven't yet gotten useful new ideas for specific problems. An useful AI would maybe be capable of mathematical simulation coupled with real world data it could request. We're still quite far..
13/18
kiteet.bsky.social @kiteet.bsky.social
I've been pestering it about questions related to ultra low temperature physics (I work at a dilution refrigerator company), and it's kind of obvious that no amount of intelligence can make noteworthy discoveries. The required materials properties data does not exist or is specific to situation.1/2
14/18
Singularity's Bounty e/
@catblanketflower.yuwakisa.com
I guess we all sort of assumed that when AI starts beating the curse of dimensionality and branching out into latent space, it would just naturally find ASI
But once it passes a certain boundary, how do we know it's not a grand hallucination?
Verifying ASI may take more time than the Universe has
15/18
thebaldcoder.bsky.social @thebaldcoder.bsky.social
Can a pig verify Einstein's theory of relativity? Neither will humanity be able to verify a theory developed by an ASI (which it will inevitably act upon). Which is why the so-called 'doomers' are right to be concerned by all this.
16/18
Singularity's Bounty e/ @catblanketflower.yuwakisa.com
I think that's a little stark ;) Certainly will expand the boundary of human-understandable knowledge, but there may be some gains that will be indecipherable for a while
Also don't neglect the effect of AI tutoring "lifting" Mankind into better understanding of things
It's just going to be weird
17/18
Mikhail Kats @mickeykats.bsky.social
I've tried it on a number of exam problems in my graduate class and sometimes it does great and other times it makes subtle mistakes
Still very useful, but I doubt it's ready to find novel things in the field.
I do think it'll get closer to that when it can use code interpreter
18/18
Gmack @gmac65.bsky.social
Sounds like the stuff progress is made of.
To post tweets in this format, more info here: https://www.thecoli.com/threads/tips-and-tricks-for-posting-the-coli-megathread.984734/post-52211196
Ethan Mollick
I have now had multiple instances where professors have told me that they are pretty sure that o1 found something novel in their field, but that the results are technical, non-obvious and complex enough that they can't be sure without considerable checking. An interesting problem.
2/18
Ethan Mollick @emollick.bsky.social
To be clear, these aren’t earth-shattering discoveries. You can’t ask for “a cure for cancer” or “a new form of physics” and get anything real. These are narrow problems in fields like economics and computer science, and require experts to identify and frame them.
3/18
Drew Curtis @drewcurtis.bsky.social
still important
4/18
猫好きな人 @valoisdubins.bsky.social
What sorts of fields?
5/18
Ethan Mollick @emollick.bsky.social
economics and computer science
6/18
猫好きな人 @valoisdubins.bsky.social
Are these theoretical innovations? Or analyses of data?
7/18
jalex32x.bsky.social @jalex32x.bsky.social
What is the significance of finding something novel independent of it being something important? My Roomba can reach places I don’t bother to vacuum. Can o1 solve alpha fold-like problems?
8/18
Lukas Bergstrom @lukas.blue
That's interesting, because my heuristic for when AI is useful is whenever generating an answer is expensive and checking it is easy.
9/18
Ethan Mollick @emollick.bsky.social
It was until o1
10/18
Lukas Bergstrom @lukas.blue
I'm not sure o1's accuracy has improved enough to make the ROI positive for expensive-to-check answers. If it has, that would be an ever bigger deal than o1's ability to tackle bigger, harder problems.
11/18
thebaldcoder.bsky.social @thebaldcoder.bsky.social
Regardless of using AI or not I'd expect you to check the answer or in software parlance "debug it". AI is just saving you the first step of generating the answer from scratch: you near instantly have something to work with. That alone is massive.
12/18
kiteet.bsky.social @kiteet.bsky.social
Really the only way to verify anything is to do tests with specific samples under specific circumstances. I haven't yet gotten useful new ideas for specific problems. An useful AI would maybe be capable of mathematical simulation coupled with real world data it could request. We're still quite far..
13/18
kiteet.bsky.social @kiteet.bsky.social
I've been pestering it about questions related to ultra low temperature physics (I work at a dilution refrigerator company), and it's kind of obvious that no amount of intelligence can make noteworthy discoveries. The required materials properties data does not exist or is specific to situation.1/2
14/18
Singularity's Bounty e/
@catblanketflower.yuwakisa.comI guess we all sort of assumed that when AI starts beating the curse of dimensionality and branching out into latent space, it would just naturally find ASI
But once it passes a certain boundary, how do we know it's not a grand hallucination?
Verifying ASI may take more time than the Universe has
15/18
thebaldcoder.bsky.social @thebaldcoder.bsky.social
Can a pig verify Einstein's theory of relativity? Neither will humanity be able to verify a theory developed by an ASI (which it will inevitably act upon). Which is why the so-called 'doomers' are right to be concerned by all this.
16/18
Singularity's Bounty e/
@catblanketflower.yuwakisa.comI think that's a little stark ;) Certainly will expand the boundary of human-understandable knowledge, but there may be some gains that will be indecipherable for a while
Also don't neglect the effect of AI tutoring "lifting" Mankind into better understanding of things
It's just going to be weird
17/18
Mikhail Kats @mickeykats.bsky.social
I've tried it on a number of exam problems in my graduate class and sometimes it does great and other times it makes subtle mistakes
Still very useful, but I doubt it's ready to find novel things in the field.
I do think it'll get closer to that when it can use code interpreter
18/18
Gmack @gmac65.bsky.social
Sounds like the stuff progress is made of.
To post tweets in this format, more info here: https://www.thecoli.com/threads/tips-and-tricks-for-posting-the-coli-megathread.984734/post-52211196
1/11
@emollick
I have now had multiple instances where professors have told me that they are pretty sure that o1 found something novel in their field, but that the results are technical, non-obvious and complex enough that they can't be sure without considerable checking
Interesting challenge
2/11
@emollick
Very mathy economics and computer science problems, for what it is worth
3/11
@emollick
To be clear, these aren’t earth-shattering discoveries you can’t ask for “a cure for cancer” or “a new form of physics” and get anything real. These are narrow problems, and require experts to identify and frame them.
4/11
@nikseeth
This is correct, except with proper plumbing can be achieved with pre-o1-class models. At this point we are having to routinely study novel cyber related outputs to understand them despite being subject matter experts.
5/11
@MattyBoySwag143
How much of a productivity bonus is AI if I have to constantly double-check everything it writes for fear of a hallucination?
6/11
@8teAPi
The issue is going to be if it’s wrong and it drives them into time wasting.
Would be totally annoying
7/11
@huntharo
My theory is: AI doesn’t need to invent stuff. It needs to show us what we’ve already invented and haven’t been able to surface to the right people.
8/11
@DirkBruere
It does come up with some surprising "insights". However, a lot of it I suspect is "lost knowledge" from neglected papers decades old
9/11
@expectiles
I have now had multiple instances where people speculated about things that would be novel in their field, but the solutions were technical, non-obvious and complex enough that they can't be sure without considerable checking.
Regular workflow.
10/11
@d_bachman21
Fascinating! Something to watch for sure
11/11
@unclecode
We rely on these human expert or I like to call them ‘biological very large language models’, to confirm AI’s novel insights or in general novelty. How do we even know we know the novelty? Anyway we are all we have. But if advanced systems start defining ‘new’ without our input, where does that leave us as judges? It’s a shift I’m exploring.
To post tweets in this format, more info here: https://www.thecoli.com/threads/tips-and-tricks-for-posting-the-coli-megathread.984734/post-52211196
@emollick
I have now had multiple instances where professors have told me that they are pretty sure that o1 found something novel in their field, but that the results are technical, non-obvious and complex enough that they can't be sure without considerable checking
Interesting challenge
2/11
@emollick
Very mathy economics and computer science problems, for what it is worth
3/11
@emollick
To be clear, these aren’t earth-shattering discoveries you can’t ask for “a cure for cancer” or “a new form of physics” and get anything real. These are narrow problems, and require experts to identify and frame them.
4/11
@nikseeth
This is correct, except with proper plumbing can be achieved with pre-o1-class models. At this point we are having to routinely study novel cyber related outputs to understand them despite being subject matter experts.
5/11
@MattyBoySwag143
How much of a productivity bonus is AI if I have to constantly double-check everything it writes for fear of a hallucination?
6/11
@8teAPi
The issue is going to be if it’s wrong and it drives them into time wasting.
Would be totally annoying
7/11
@huntharo
My theory is: AI doesn’t need to invent stuff. It needs to show us what we’ve already invented and haven’t been able to surface to the right people.
8/11
@DirkBruere
It does come up with some surprising "insights". However, a lot of it I suspect is "lost knowledge" from neglected papers decades old
9/11
@expectiles
I have now had multiple instances where people speculated about things that would be novel in their field, but the solutions were technical, non-obvious and complex enough that they can't be sure without considerable checking.
Regular workflow.
10/11
@d_bachman21
Fascinating! Something to watch for sure
11/11
@unclecode
We rely on these human expert or I like to call them ‘biological very large language models’, to confirm AI’s novel insights or in general novelty. How do we even know we know the novelty? Anyway we are all we have. But if advanced systems start defining ‘new’ without our input, where does that leave us as judges? It’s a shift I’m exploring.
To post tweets in this format, more info here: https://www.thecoli.com/threads/tips-and-tricks-for-posting-the-coli-megathread.984734/post-52211196
P4L
