OpenAI Just Signed the Death of LangChain

OpenAI's latest update to its API has introduced a groundbreaking feature: Functions. In essence, these functions ensure that responses to API calls follow a specific JSON structure, a capability that opens up numerous possibilities, such as interacting with other APIs, converting natural language into API calls, and extracting structured data from text. Such advancements in technology often mean other tools become less relevant or even obsolete. This time, it seems LangChain and Llama Index are the ones being left behind. However, it's not all roses. The introduction of functions comes with its own set of costs and challenges that users should be aware of.

Functions: The Game Changer in OpenAI's API Update

Functions in the context of OpenAI's API are a feature that enables the assurance of a particular JSON structure in the response to an API call. This structuring is done by injecting these functions into the system message, using a syntax that the model has been trained on. The ability to format responses in such a way greatly enhances the interaction with other APIs, converting natural language into API calls, and extracting structured data from text.

Such capabilities could arguably make some functionalities of LangChain and Llama Index obsolete. These tools, which were designed to provide language-related services and to index information respectively, may not be as efficient or useful given the new capabilities introduced by OpenAI.

The Hidden Costs of Using Functions

While the introduction of functions in OpenAI's API provides significant advantages, it also introduces new considerations that developers and users need to account for. Functions count against the model's context limit and are billed as input tokens, which could lead to increased costs when using OpenAI's GPT models. For users who frequently encounter context limits, it might be beneficial to limit the number of functions used or reduce the length of the documentation provided for function parameters.

Moreover, the model might generate invalid JSON or hallucinate parameters of your function, and users do not have control over how the system message is worded behind the scenes, which limits the ability to optimize this aspect.

A Walkthrough of OpenAI's Functions: An Example

To better understand the application of functions in the OpenAI API, let's consider a code sample. In this example, we'll illustrate how the OpenAI API can be used to fetch Wikipedia article summaries relevant to a user's question.

Firstly, we set up the OpenAI API key:

from apiKey import apikey
import os
import openai
import json
import wikipedia

os.environ["OPENAI_API_KEY"] = apikey
openai.api_key = apikey

Next, we define a function to get Wikipedia article summaries for a given subject:

def get_wikipedia_article_summaries(subject, limit=2):
    """Get a list of summaries of Wikipedia articles about a certain subject."""
    print(f"Fetching summaries for subject: {subject}")
    results = []
    try:
        articles = wikipedia.search(subject, results=limit)
        for article in articles:
            try:
                summary = wikipedia.summary(article) 
                results.append({"title": article, "summary": summary})
            except wikipedia.exceptions.DisambiguationError as e:
                print(f"Disambiguation error for {article}: {e}")
            except wikipedia.exceptions.PageError as e:
                print(f"Page error for {article}: {e}")
            except Exception as e:
                print(f"An error occurred: {e}")
        results = str(e)

    return json.dumps({"subject": subject, "summaries": results})

This function fetches summaries of Wikipedia articles on a specified subject. It will catch and handle various exceptions that might occur during the process.

Following this, we create a function that uses the OpenAI API to answer a user's question:

def answer_question(question):
    print(f"Received question: {question}")
    response = openai.ChatCompletion.create(
        model="gpt-3.5-turbo-0613",
        messages=[{
            "role": "user",
            "content": f"What subject should we search on wikipedia to answerthis question: {question}?"
        }],
        functions=[{
            "name": "get_wikipedia_article_summaries",
            "description": "Get summaries of articles relevant to the question",
            "parameters": {
                "type": "object",
                "properties": {
                    "subject": {
                        "type": "string",
                        "description": "The subject term that will be searched on wikipedia",
                    },
                    "limit": {
                        "type": "integer",
                        "description": "the number of summaries to be obtained from wikipedia to answer the question"
                    },
                },
                "required": ["subject"],
            },
        }],
        function_call={"name": "get_wikipedia_article_summaries"},
    )

    message = response["choices"][0]["message"]

    if message.get("function_call"):
        function_name = message["function_call"]["name"]
        arguments = json.loads(message["function_call"]["arguments"])
        subject = arguments["subject"]

        result = json.loads(get_wikipedia_article_summaries(subject))
        article_summaries = result["summaries"]

        article_summaries_str = " ".join([
            f'Title: {summary["title"]}, Summary: {summary["summary"]}'
            for summary in article_summaries
        ])

        second_response = openai.ChatCompletion.create(
            model="gpt-3.5-turbo-0613",
            messages=[
                {
                    "role": "user",
                    "content": question
                },
                message,
                {
                    "role": "function",
                    "name": function_name,
                    "content": article_summaries_str,
                },
            ],
        )

    return second_response["choices"][0]["message"]["content"]

This function first determines the relevant subject to search for on Wikipedia by using the OpenAI API. It then fetches the relevant Wikipedia article summaries using the earlier defined function. Lastly, it sends a second request to the OpenAI API to construct an answer from the fetched summaries.

Finally, we create a loop to continuously receive and answer user questions:

while True:
    user_question = input("Enter your question or 'exit' to stop the program:\n\n")

    if user_question == 'exit':
        break

    print(answer_question(user_question))

This loop allows for continual interaction until the user inputs 'exit'.

In essence, this example demonstrates how the OpenAI API, with its new functions feature, can be used to facilitate natural language processing tasks and information extraction, capabilities that were previously associated with tools like LangChain.

In conclusion, the introduction of functions in the OpenAI API is a momentous development that greatly enhances the capabilities of the API, potentially relegating some functionalities of LangChain and Llama Index to the sidelines. However, the associated increase in costs and potential for generating invalid JSON or hallucinated function parameters should also be considered when incorporating this new feature into your development workflow.