Watching language model tooling slowly mature, it is interesting to see a progressive constraining of capabilities.
Historically, programming languages have become more abstract (”higher-lever”) over time:
Assembly → C → Python
With language models, we may have arrived at the highest possible level of abstraction—natural language—and now we are beginning to wrap back around the other way.
A high level of abstraction is great in that it lowers the barrier to entry for programming but it comes with the cost of increased ambiguity. Sure, your compiler can now try to guess your intentions but that doesn’t mean you would always like it to do that.
Even more important is the fact that language models are non-deterministic. That is, each successive time you run your “program” you might receive a different output.
This is a huge problem, almost a non-starter when it comes to integrating LLMs into traditional programming pipelines. That is why so much research has gone into making LLMs reliably output a more constrained set of tokens that can be validated according to a predetermined schema.
JSONformer, GPT-JSON, and Guidance are all examples of prior work along these lines.
Well, earlier this week OpenAI announced a new API endpoint that points to models they finetuned for exactly this purpose.
Developers can now describe functions to gpt-4-0613 and gpt-3.5-turbo-0613, and have the model intelligently choose to output a JSON object containing arguments to call those functions. This is a new way to more reliably connect GPT’s capabilities with external tools and APIs.
I can’t wait to see what people are able to accomplish using these new capabilities.