Updating and enhancing the BibTeX fetcher to V2

This major update to the BibTeX Manager introduces a visual interface, SemanticScholar as the backend and a new logo, designed by Ngoc-Han Le. On top of that, the entire codebase was rewritten to iron out the inconsistencies that version 0.3.0 had. All these changes were done to make this extension integrate into the LaTeX writing workflow for VSCode users.

Fetching A Citation

In version 0.3.0, users had to copy the paper's title to fetch its BibTeX. They clicked on a button labeled "Add Citation" and waited for the fetching algorithm. The system would first search DBLP and then another source if unsuccessful. If the extension didn't find the right citation, users had to search for it online. Sometime in mid-August 2023, DBLP's API gave back HTML when it encountered an error. This meant users could accidentally add an entire HTML page to their library document. This made the extension highly unpredictable and made it clear that the extension needed an overhaul.

Now, with the updated version, users paste the title into the search input of the extension and click " Search" . The extension then displays a list of up to 5 matching papers. Users can then select the correct one by clicking the "GetBibTeX" button. The video below demonstrates how rapidly the papers are located and the BibTeX is added.

Behind The Scenes: Fetching Logic

The core functionality of the BibTeX Manager extension revolves around its ability to fetch BibTeX citations from Semantic Scholar. This is achieved through a series of steps:

1. Activating the Extension: When the extension is activated, it registers a new SearchBTViewProvider which provides the main interface for searching and fetching BibTeX citations. The addCitation command is also registered, which triggers the search functionality.
   See the code here
2. Searching for a Paper: The SearchBTViewProvider class handles the logic for searching papers. When a user inputs a paper title, the extension sends a message to the webview provider to initiate the search. The search request is made to Semantic Scholar's API, which returns a list of matching papers. In the code below the communication between the webview and the webview provider is shown. To give the user feedback about the current state of the extension there is also a loading bar displayed while the Semantic Scholar's API is called.

webviewView.webview.onDidReceiveMessage(data => {
    switch (data.type) {
        case 'searchPaper':
            webviewView.webview.postMessage({ type: 'loading' });
            const paperTitle = data.message;
            fetchSemanticScholar(paperTitle, 5).then(data => {
                webviewView.webview.postMessage({ type: 'papers', message: data.data });
                webviewView.webview.postMessage({ type: 'stopLoading' });
            }).catch(err => {
                console.error(err);
                webviewView.webview.postMessage({ type: 'stopLoading' });
            });
            break;
    }
});

1. Displaying the Results: The first API call will return a list of up to 5 entries. These are then sent to the webview as a message. The webview then displays the papers in a list, each with the option "Get BibTeX".
2. Fetching BibTeX Citation: Once the user selects a paper from the search results, the extension fetches the corresponding BibTeX citation using the paper's unique ID. This is done using the fetchBibTeX function, which sends a request to Semantic Scholar's API to retrieve the BibTeX citation. In the returned JSON file, SemanticScholar already includes the BibTeX code for citation. The returned BibTeX is then written to a library.bib file which can be specified by the user inside the VSCode settings.

export function fetchBibTeX(paperId: string): Promise<string | undefined> {
    return new Promise<string | undefined>((resolve, reject) => {
        const url = new URL(`https://api.semanticscholar.org/graph/v1/paper${paperId}`);
        url.searchParams.append('fields', 'citationStyles');
        https.get(url, (res) => {
            let data = '';
            res.on('data', (chunk) => {
                data += chunk;
            });
            res.on('end', () => {
                const jsonData = JSON.parse(data);
                if (jsonData.citationStyles && jsonData.citationStyles.bibtex) {
                    resolve(jsonData.citationStyles.bibtex);
                } else {
                    resolve(undefined);
                }
            });
        }).on('error', (error) => {
            reject(error);
        });
    });
}

After the BibTeX was written to the library its ready to be used in the latex project.

About The Future Of BibTeX Manager

Currently the BibTeX Manager only supports papers listed on SemanticScholar. We are planning to add the capabilities of generating BibTeX for Websites and Books too. There is also a new project we are working on which will be a surprise for a future blog post.

We hope you will get as much joy out of this simple extension as us. If you are using BibTeX Manager and you want to add changes or request features please feel free to contact me at matthias.anton.schedel@gmail.com. Also check out the GitHub repository if you are interested.

Happy writing!