VIM-Post Request to a GPT Text Generation Web API

VRC is a Vim plug-in which enables sending requests to and display responses from RESTful services inside Vim buffers.

A web application endpoint collects the JSON request which includes the initial prompt and the parameters for the NLP model. It then sends back a JSON response with the generated text and the number of tokens contained in the initial prompt.

The Python web application (bottlepy framework) where the endpoint is defined uses Hugging Face's pre-trained machine-learning library to find the number of tokens in the prompt and to generate the additional text.

from transformers import pipeline, set_seed, GPT2TokenizerFast
...
@post('/gpt/testprompt') 
def testprompt():
    prompt = request.json.get('prompt')
    set_seed(request.json.get('randomized'))

    # https://huggingface.co/docs/transformers/v4.18.0/en/main_classes/pipelines#transformers.pipeline
    generator = pipeline("text-generation", model="gpt2")
    tokenizer = GPT2TokenizerFast.from_pretrained("gpt2")
   
    n_tokens = len(tokenizer(prompt)["input_ids"])
    data = {"prompt": prompt, "tokens": n_tokens}
    gp = generator(prompt, max_length=request.json.get('max_length'), 
            num_return_sequences = 1,
            top_p = request.json.get('top_p'),
            do_sample=True, temperature= request.json.get('temperature'))[0]
    l = gp["generated_text"] 
    data["output"]=l    
    return data

The video below shows an example of a request created in a VIM VRC buffer. The output produced with the NLP model and the number of tokens in the initial prompt will be displayed in a new vertical buffer.