Online File Searcher
The purpose of this project is to give a user the ability to upload files (PDFs, text files and audio files as well as URLs and YouTube videos with transcriptions) and then be able to search them using Open AI embeddings. Using Langchain's Node.js package, I have the user upload a file from my website which then goes to an S3 bucket. From there, it gets processed by a Lambda which then converts it to text, slices the text up into 500 character chunks, creates embeddings from it and then inserts it into a vector database. Then, in the website I created, the user can then query the vector database that will show them which of the files they uploaded, and if it is a PDF or text file, what page it most closely answers t...