Humans have been trained over the last few decades to expect certain things from information technology. To list a few:
- Hardware and software will improve (in such metrics as performance, user experience, and reliability) at a Moore’s-law type of pace, before transitioning to more incremental improvement.
- Individual software tools can reliably produce high-quality outputs, but the input data must be of the highest quality, carefully formatted in the specific way that the tool demands.
- The more advanced the tool, the more complex the specifications and edge cases, making interoperability between tools (particularly between different providers) a significant technical challenge unless well-designed standards are in place.
- The humans will make all the key executive decisions; the software tool influences the decision-making process through its success or failure in executing human-directed orders.
All of these expectations will need to be recalibrated, if not abandoned entirely, with the advent of generative AI tools such as GPT-4. These tools perform extremely well with vaguely phrased (and slightly erroneous) natural language prompts, or with noisy data scraped from a web page or PDF. I could feed GPT-4 the first few PDF pages of a recent math preprint and get it to generate a half-dozen intelligent questions that an expert attending a talk on the preprint could ask. I plan to use variants of such prompts to prepare my future presentations or to begin reading a technically complex paper. Initially, I labored to make the prompts as precise as possible, based on experience with programming or scripting languages. Eventually the best results came when I unlearned that caution and simply threw lots of raw text at the AI. This level of robustness may enable AI tools to integrate with traditional software tools—or with each other, or with personal data and preferences. It will disrupt workflows everywhere in a way that the current AI tools, used in isolation, merely hint at doing.