Search Engine & Indexer
In this project, my team and I created a search engine and indexer from scratch. We were able to create a search engine that can take in a query and return websites in under a second using Boolean Searching.
Example of using the search engine with the query of “master of software”
In order to create a search engine, we needed to create an indexer that can scrape websites for words, in order to return an inverted index that we can then use in the search engine in order to successfully return a query. We used a collection of over 55,000 UCI ICS websites as the websites we were going to index and search over.
After creating a successful indexer that parsed through every website and created tokens for each alphanumeric word, we began to create the search engine for the indexer. We did this by writing code that would take in a users input, and then through many methods would return the websites that contained all of the words in the user’s query.
From this project I learned how important speed is to a program, especially in order to keep up with how fast the world is getting. I also gained more experience using python as well as coding libraries such as Beautiful Soup and This project was all done with pair programming with three members each taking turns to be the one coding.